A New Multi-Agent Reinforcement Learning Method Based on Evolving Dynamic Correlation Matrix

摘要

Multi-agent reinforcement learning approaches can be roughly classified into two categories. One is the agent-based approach which can be implemented in real distributed systems, though most approaches of this type cannot provide meaningful theoretical verifications. The other can be seen as the more formalized approach, which can provide theoretical results. However, most of current algorithms usually require unrealistic global communication, which makes them impractical for real distributed systems. In this article, we propose a dynamic correlation matrix based multi-agent reinforcement learning approach where the meta-parameters are evolved using an evolutionary algorithm. We believe that our approach is able to fill the gap between the two kinds of traditional multi-agent reinforcement learning approaches by providing both agent-level implementation and system-level convergence verification. The basic idea of this approach is that agents learn not only from local environmental feedback, i.e., their own experiences and rewards, but also from other agents' experiences. In this way, the agents' learning speed can be increased significantly. The performance of the proposed algorithm is demonstrated on a number of application scenarios, including blackjack games, urban traffic control systems and multi-robot foraging.