TDE2-MBRL: Energy-exchange Dynamics Learning with Task Decomposition for Spring-loaded Bipedal Robot Locomotion

Abstract

Spring-loaded Inverted Pendulum (SLIP) inspired bipedal robots (SLIP-biped) have high agility owing to their fault tolerance under impacts. Controlling a SLIP-biped requires capturing its dynamics; however, its high complexity makes analytic method implementation challenging. Thus, a Model-based Reinforcement Learning (MBRL) that learns a dynamics model and utilizes it for control design appears to be a reasonable alternative. Nevertheless, modeling high complexity dynamics with conventional MBRL approaches requires enormous samples or a high computation load. Therefore, exploring a simplified and compact dynamics model for SLIP-biped would be a key to increasing the feasibility of MBRL implementation and real-time control. We propose a Task-Decomposed Energy-exchange dynamics learning with MBRL (TDE2-MBRL) to capture simplified SLIP-biped dynamics and utilize them for control. Specifically, under the law of energy conservation, we model the energy exchange to reduce dynamics' dimensionality. Next, we decompose the SLIP-biped dynamics into locomotion task phases to cope with dynamics dissimilarity. The effectiveness is demonstrated by hopping skill acquisition with a precise simulated SLIP-biped replica of a real SLIP-biped. The experiment results show that TDE2-MBRL improves learning efficiency and control frequency while having comparable model accuracy to the standard MBRL.