Comparison of Reinforcement Learning Algorithms for Motion Control of an Autonomous Robot in Gazebo Simulator

Abstract

This article compares various implementations of deep Q learning as it is one of the most efficient reinforcement learning algorithms for discrete action space systems. The efficiency of the implementations for the classical Cartpole problem ported to the Gazebo environment is investigated. Then, these algorithms are compared for a self-created bipedal robot problem. Since the creation and configuration of a real robotic system is a laborious process, the initial debugging of the robot can be performed using the appropriate software that simulates the real environment. In our case, the Gazebo simulator was used. Using the simulator allows you to conduct research without having a real robotic system. In this case, it is possible to transfer the results from the simulator to the real system. The result of the study is the conclusion about the greatest efficiency of deep Q-learning with the experience reproduction mechanism. Also, the conclusion is that even for a robot with two degrees of freedom, Q-learning algorithms are not effective enough, and a comparative study with other families of reinforcement learning algorithms is needed.