A Survey of Cooperative Multi-Agent Reinforcement Learning for Multi-Task Scenarios

摘要

Cooperative multi-agent reinforcement learning (MARL) is a key technology for enabling cooperation in complex multi-agent systems. It has achieved remarkable progress in areas such as gaming, autonomous driving, and multi-robot control. Empowering cooperative MARL with multi-task decision-making capabilities is expected to further broaden its application scope. In multi-task scenarios, cooperative MARL algorithms need to address 3 types of multi-task problems: reward-related multi-task, arising from different reward functions; multi-domain multi-task, caused by differences in state and action spaces, state transition functions; and scalability-related multi-task, resulting from the dynamic variation in the number of agents. Most existing studies focus on scalability-related multi-task problems. However, with the increasing integration between large language models (LLMs) and multi-agent systems, a growing number of LLM-based multi-agent systems have emerged, enabling more complex multi-task cooperation. This paper provides a comprehensive review of the latest advances in this field. By combining multi-task reinforcement learning with cooperative MARL, we categorize and analyze the 3 major types of multi-task problems under multi-agent settings, offering more fine-grained classifications and summarizing key insights for each. In addition, we summarize commonly used benchmarks and discuss future directions of research in this area, which hold promise for further enhancing the multi-task cooperation capabilities of multi-agent systems and expanding their practical applications in the real world.