A Curiosity Algorithm for Robots Based on the Free Energy Principle

Abstract

In this paper, we propose a curiosity algorithm for sample efficient data collection in robotic task learning. Reinforcement learning (RL) is one of the most commonly used algorithms for robotic task learning. In general, RL agents explore the environment by random actions. However, random exploration results in poor sample efficiency when the state-action space is continuous and high-dimensional, such as in a robotic object manipulation task. By making the robot curious, the robot would become interested in the unknown region and explore that region. In order to make the robot curious, we utilize the free energy principle, which has been attracting attention in the field of neuroscience and AI. We propose a curiosity algorithm that combines with Soft Actor-Critic and show that it is naturally derived from the free energy principle. Moreover, we verify the effectiveness of the proposed algorithm in simulation and real-world experiments of robotic object manipulation task learning.