A Deep Safe Reinforcement Learning Approach for Mapless Navigation

Abstract

Reinforcement learning (RL) is used more and more in robot navigation, however the safety of RL is usually not guaranteed. To improve the safety in the end-to-end mapless navigation using deep reinforcement learning (DRL), we propose a deep safe RL approach which uses a safe RL algorithm called Constrained Policy Optimization (CPO) and design the Actor-Critic-Safety (ACS) architecture to apply CPO. We use the Social Force Pedestrian Simulator based on social force model to simulate the dynamic environment with pedestrians in Gazebo. Experiment results show that the proposed approach can obviously increase the success rate and reduce the collision rate, which means the safety in navigation is improved. The planned path is almost as good as by ROS move_base which needs to build a map of environment first. What’s more, the model trained in static environment is able to generalize to unseen dynamic environment with pedestrians without any fine tuning and behaves well.