Motion Coordination of Multiple Robots Based on Deep Reinforcement Learning

Abstract

Multi-robot motion coordination is a sequential decision problem that controls each robot's action through a coordination mechanism to avoid collisions and reach their respective destinations. This problem can be viewed as a Markov Decision Process. Therefore, under determinate conditions of the motion path, we make the attempt of applying deep reinforcement learning (DRL) to solve the motion coordination problems. In this paper, in order to make this problem solvable through RL methods, we design a Path Checkerboard Diagram (PCD) to represent the collision relationship among different paths and determine whether collisions occur when the agent takes action. Technically, we design the states, actions and reward function. However, the traditional Deep Q-Networks (DQN) algorithm has the problem of overestimation of state-action values and high approximation error variance of the target values. To fit the motion coordination problem scenario, we further propose a new algorithm Multi-Loss Double DQN (MLDDQN) based on the Double DQN (DDQN), which minimizes the weighted sum of loss of different target networks we have learned before. We evaluate the performance of our algorithm through the different motion coordination tasks. Results suggest that our MLDDQN method yields remarkable improvements in terms of the success rate, success steps and the stability of training process compared with the original DDQN algorithm.