Deep Reinforcement Learning of Navigation in a Complex and Crowded Environment with a Limited Field of View

Abstract

Mobile robots are required to navigate freely in a complex and crowded environment in order to provide services to humans. For this navigation ability, deep reinforcement learning (DRL)-based methods are gaining increasing attentions. However, existing DRL methods require a wide field of view (FOV), which imposes the usage of high-cost lidar devices. In this paper, we explore the possibility of replacing expensive lidar devices with affordable depth cameras which have a limited FOV. First, we analyze the effect of a limited field of view in the DRL agents. Second, we propose a LSTM agent with Local-Map Critic (LSTM-LMC), which is a novel DRL method to learn efficient navigation in a complex environment with a limited FOV. Lastly, we introduce the dynamics randomization technique to improve the robustness of the DRL agents in the real world. We found that our method with a limited FOV can outperform the methods having a wide FOV but limited memory. We provide the empirical evidence that our method learns to implicitly model the surrounding environment and dynamics of other agents. We also show that a robot with a single depth camera can navigate through a complex real-world environment using our method.