Interactive Reinforcement Learning With Bayesian Fusion of Multimodal Advice

Abstract

Interactive Reinforcement Learning (IRL) has shown promising results in decreasing the learning times of Reinforcement Learning algorithms by incorporating human feedback and advice. In particular, the integration of multimodal feedback channels such as speech and gestures into IRL systems can enable more versatile and natural interaction of everyday users. In this letter, we propose a novel approach to integrate human advice from multiple modalities into IRL algorithms. For each advice modality we assume an individual base classifier that outputs a categorical probability distribution and fuse these distributions using the Bayesian fusion method Independent Opinion Pool. While existing approaches rely on heuristic fusion, our Bayesian approach is theoretically founded and fully exploits the uncertainty represented in the distributions. Experimental evaluations in a simulated grid world scenario and on a real-world human-robot interaction task with a 7-DoF robot arm show that our method clearly outperforms the closest related approach for multimodal IRL. In particular, our novel approach is more robust against misclassifications of the modalities’ individual base classifiers.