A Hierarchical Reinforcement Learning Framework based on Soft Actor-Critic for Quadruped Gait Generation

Abstract

Recently, reinforcement learning has become a promising control method of legged robot. However, it is challenging to train from scratch which requires perfect networks and reward design. In this paper, a hierarchical reinforcement learning framework based on Soft Actor-Critic has been proposed to find the appropriate gait of quadruped robot in the environment. The framework is composed of a low-level policy for generating joint reference trajectory and a high-level policy for gait optimization. In low-level policy, we use radial basis network and evolutionary computation solver to change the shape of reference trajectory in order to search for a better reference trajectory. In high-level policy, joint angle increment is learned to optimize gait. The experimental results show that the hierarchical framework is better than that of using Soft Actor-Critic only.