Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement\n Learning

摘要

In this work, we present and study a training set-up that achieves fast\npolicy generation for real-world robotic tasks by using massive parallelism on\na single workstation GPU. We analyze and discuss the impact of different\ntraining algorithm components in the massively parallel regime on the final\npolicy performance and training times. In addition, we present a novel\ngame-inspired curriculum that is well suited for training with thousands of\nsimulated robots in parallel. We evaluate the approach by training the\nquadrupedal robot ANYmal to walk on challenging terrain. The parallel approach\nallows training policies for flat terrain in under four minutes, and in twenty\nminutes for uneven terrain. This represents a speedup of multiple orders of\nmagnitude compared to previous work. Finally, we transfer the policies to the\nreal robot to validate the approach. We open-source our training code to help\naccelerate further research in the field of learned legged locomotion.\n