Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance\n Action Space

Abstract

Intelligent agents must be able to think fast and slow to perform elaborate\nmanipulation tasks. Reinforcement Learning (RL) has led to many promising\nresults on a range of challenging decision-making tasks. However, in real-world\nrobotics, these methods still struggle, as they require large amounts of\nexpensive interactions and have slow feedback loops. On the other hand, fast\nhuman-like adaptive control methods can optimize complex robotic interactions,\nyet fail to integrate multimodal feedback needed for unstructured tasks. In\nthis work, we propose to factor the learning problem in a hierarchical learning\nand adaption architecture to get the best of both worlds. The framework\nconsists of two components, a slow reinforcement learning policy optimizing the\ntask strategy given multimodal observations, and a fast, real-time adaptive\ncontrol policy continuously optimizing the motion, stability, and effort of the\nmanipulator. We combine these components through a bio-inspired action space\nthat we call AFORCE. We demonstrate the new action space on a contact-rich\nmanipulation task on real hardware and evaluate its performance on three\nsimulated manipulation tasks. Our experiments show that AFORCE drastically\nimproves sample efficiency while reducing energy consumption and improving\nsafety.\n