Decentralized Multi-Robot Navigation in Unknown Environments via Hierarchical Deep Reinforcement Learning

Abstract

Multi-robot navigation in complex scenarios such as container terminalss is a challenging problem where each robot can only perceive a subset of the states and intentions of other robots. In this paper, we propose a multi-robot navigation framework based on option-based hierarchical deep reinforcement learning (DRL) for rapid and safe navigation. The framework comprises two control models: a low-level model that generates actions using sub-policies, and a high-level model that learns a stable and reliable behavior selection policy automatically. Additionally, we design a PID-based target drive controller and an emergency braking controller to enhance obstacle avoidance efficiency and generalization ability in hazardous scenarios. We evaluate the proposed method against existing DRL-based navigation methods in various simulated scenarios with thorough performance evaluations. Our results indicate that the proposed framework significantly improves multi-robot navigation performance in complex scenarios and exhibits excellent generalization ability to new scenarios.