A Survey of Maximum Entropy-Based Inverse Reinforcement Learning: Methods and Applications

摘要

In recent years, inverse reinforcement learning algorithms have garnered substantial attention and demonstrated remarkable success across various control domains, including autonomous driving, intelligent gaming, robotic manipulation, and automated industrial systems. Nevertheless, existing methodologies face two persistent challenges: (1) finite or non-optimal expert demonstration and (2) ambiguity in which different reward functions lead to same expert strategies. To improve and enhance the expert demonstration data and to eliminate the ambiguity caused by the symmetry of rewards, there has been a growing interest in research on developing inverse reinforcement learning based on the maximum entropy method. The unique advantage of these algorithms lies in learning rewards from expert presentations by maximizing policy entropy, matching expert expectations, and then optimizing the policy. This paper first provides a comprehensive review of the historical development of maximum entropy-based inverse reinforcement learning (ME-IRL) methodologies. Subsequently, it systematically presents the benchmark experiments and recent application breakthroughs achieved through ME-IRL. The concluding section analyzes the persistent technical challenges, proposes promising solutions, and outlines the emerging research frontiers in this rapidly evolving field.