Awareness Map Enabled Semi-Centralized MARL for Multi-Robot Collaboration

Abstract

Facilitating efficient collaboration among multiple robots in complex smart environments presents significant challenges. Multi-agent reinforcement learning (MARL) offers a viable solution for training robots in task selection, navigation, and collaboration. However, classical MARL methods that rely solely on local observations result in the loss of global awareness, leading to suboptimal performance. To tackle this, many collaborative MARL methods employ centralized learning to utilize complete information during training. However, acquiring such global information in real-world multi-robot systems is often impractical and computationally expensive, necessitating an abstraction method to enhance decision-making and accelerate the learning process. In this research, we propose a novel semi-centralized MARL (SC-MARL) framework that enhances communication among robots through a shared knowledge base called the Awareness Map (AM). We introduce three variations of the Awareness Map, Uniform-AM, Linear-AM, and Sigmoid-AM, and evaluate their performance in a predefined multi-robot environment. Using Unity3D, we designed various smart environments with different complexities, incorporating diverse learning objectives and cooperative challenges. Experimental results show that our AM-enabled SC-MARL effectively trains robot groups, achieving 50% higher performance than decentralized learning methods. Additionally, the adaptability of our approach supports transfer learning, allowing robots to reuse knowledge across multiple scenarios and learn faster than starting from scratch.