Multi-Robot Cooperative Pursuit-Evasion Control: A DeepReinforcement Learning Approach based on Prioritized Experience Replay

Abstract

Cooperative pursuit systems based on traditional model control rules are less adaptable and less robust to complex dynamic environments. In this paper, we study the cooperative pursuit-evasion with collision avoidance in multi-robot systems. We first adopt the Multi-Agent Twin Delayed Deep Deterministic policy gradient (MATD3) algorithm, and design a cooperative pursuit framework that uses the information of multiple robots in the learning process to more accurately predict the actions that robots will take. Then, we propose a Prioritized Experience Replay based MATD3 (PER-MATD3) algorithm which solves the problem of sparse reward in multi-robot cooperative pursuit algorithm by adopting the higher prioritized experience data update network when sampling. Simulation results show that the proposed PER-MATD3 algorithm reduces the collisions among robots, the collisions between robots and obstacles, and the capture time by 60.97%, 68.42%, and 30.37% respectively compared with the baseline algorithms. Moreover, the PER-MATD3 algorithm improves the capture success rate by 25.71% and achieves a faster convergence speed in continuous decision-making than the baseline algorithms.