Balance Control of a Humanoid Robot Using DeepReinforcement Learning

Abstract

In this paper, a deep reinforcement learning algorithm is presented to control a humanoid robot. We have used two control levels in a hierarchical manner. Within the high-level control architecture, a policy is determined by a combination of two neural networks as actor and critic and optimized using proximal policy optimization (PPO) method. The output policy specifies reference angles for robot joint space. At the low-level control, a PID controller regulates robot states around the reference values. The robot model is provided in MuJoCo physics engine and simulations are performed using mujoco-py library. During the simulations robot could maintain its balance stability against wide variety of exerted disturbances. The results showed that the proposed algorithm had a good performance and could resist larger push impacts compared to the pure PID controller.