Solving Challenging Dexterous Manipulation Tasks With Trajectory\n Optimisation and Reinforcement Learning

Abstract

Training agents to autonomously learn how to use anthropomorphic robotic\nhands has the potential to lead to systems capable of performing a multitude of\ncomplex manipulation tasks in unstructured and uncertain environments. In this\nwork, we first introduce a suite of challenging simulated manipulation tasks\nthat current reinforcement learning and trajectory optimisation techniques find\ndifficult. These include environments where two simulated hands have to pass or\nthrow objects between each other, as well as an environment where the agent\nmust learn to spin a long pen between its fingers. We then introduce a simple\ntrajectory optimisation that performs significantly better than existing\nmethods on these environments. Finally, on the challenging PenSpin task we\ncombine sub-optimal demonstrations generated through trajectory optimisation\nwith off-policy reinforcement learning, obtaining performance that far exceeds\neither of these approaches individually, effectively solving the environment.\nVideos of all of our results are available at:\nhttps://dexterous-manipulation.github.io/\n