RMRL: Robot Navigation in Crowd Environments With Risk Map-Based Deep Reinforcement Learning

Abstract

Achieving safe and effective navigation in crowds is a crucial yet challenging problem. Recent work has mainly encoded the pedestrian-robot state pairs, which cannot fully capture the interactions among humans. Besides, existing work attempts to achieve “hard” collision avoidance, which may leave no feasible path to the robot in human-rich scenarios. We suppose that this can be addressed by introducing the local risk map and thus incorporate the risk map into the deep reinforcement learning architecture. The proposed map structure contains the crowd interaction states and geometric information. Meanwhile, a “soft” risk mapping of pedestrians is proposed to promote the robot to generate more humanlike motion patterns, and the riskaware dynamic window is designed to enhance the robot's obstacle avoidance ability. Experiments show that our method outperforms the baseline in terms of navigation performance and social attributes. Furthermore, we successfully validate the proposed policy through real-world environments.