Deep Reinforcement Learning with Gait Mode Specification for Quadrupedal Trot-Gallop Energetic Analysis

Abstract

Quadruped system is an animal-like model which has long been analyzed in terms of energy efficiency during its various gait locomotion. The generation of certain gait modes on these systems has been achieved by classical controllers which demand highly specific domain-knowledge and empirical parameter tuning. In this paper, we propose to use deep reinforcement learning (DRL) as an alternative approach to generate certain gait modes on quadrupeds, allowing potentially the same energetic analysis without the difficulty of designing an ad hoc controller. We show that by specifying a gait mode in the process of learning, it allows faster convergence of the learning process while at the same time imposing a certain gait type on the systems as opposed to the case without any gait specification. We demonstrate the advantages of using DRL as it can exploit automatically the physical condition of the robots such as the passive spring effect between the joints during the learning process, similar to the adaptation skills of an animal. The proposed system would provide a framework for quadrupedal trot-gallop energetic analysis for different body structures, body mass distributions and joint characteristics using DRL.