Behavioral Cloning from Observation with Bi-directional Dynamics Model

Abstract

Robotics is rapidly expanding its workplace from industrial factories to more complicated fields to work on behalf of human. The difficulty of programming the operations human does in advance, however, prevents this expansion. Behavioral cloning is one of the promising approaches to acquire the operations effectively from the expert's demonstrations, which consist of the states and the performed actions of the expert. However, it is intractable and/or highly expensive for robots to measure the expert's actions. Behavioral cloning from observation fills this gap and makes it possible to imitate with state-only demonstrations by inferring actions that the expert performed from an inverse dynamics model. Our goal is to improve the accuracy of this algorithm. This is done by evaluating the inferred action using an additional forward dynamics model. Specifically, we focus on the consistency in both dynamics models, which have to be bi-directional. This bi-directionality, can classify whether the inferred action is realistic or not, and can prevent wrong updates. We show the successful improvement with our new method using various simulation tasks which are typically used in benchmarks.