Robust Recovery Controller for a Quadrupedal Robot using Deep\n Reinforcement Learning

Abstract

The ability to recover from a fall is an essential feature for a legged robot\nto navigate in challenging environments robustly. Until today, there has been\nvery little progress on this topic. Current solutions mostly build upon\n(heuristically) predefined trajectories, resulting in unnatural behaviors and\nrequiring considerable effort in engineering system-specific components. In\nthis paper, we present an approach based on model-free Deep Reinforcement\nLearning (RL) to control recovery maneuvers of quadrupedal robots using a\nhierarchical behavior-based controller. The controller consists of four neural\nnetwork policies including three behaviors and one behavior selector to\ncoordinate them. Each of them is trained individually in simulation and\ndeployed directly on a real system. We experimentally validate our approach on\nthe quadrupedal robot ANYmal, which is a dog-sized quadrupedal system with 12\ndegrees of freedom. With our method, ANYmal manifests dynamic and reactive\nrecovery behaviors to recover from an arbitrary fall configuration within less\nthan 5 seconds. We tested the recovery maneuver more than 100 times, and the\nsuccess rate was higher than 97 %.\n