Learning Robot Motion in a Cluttered Environment Using Unreliable Human Skeleton Data Collected by a Single RGB Camera

Abstract

Existing learning from demonstration (LfD) frameworks have difficulty dealing with unreliable and limited number of demonstrations. To address this issue, we proposed a new motion planning framework named experience-driven random tree connect with human demonstration (ERTC-HD), which can facilitate the identification of valid motions in cluttered environments by only using human skeleton information extracted from a single red, green, and blue (RGB) camera. The concept of this framework is to only extract the comprehensive features of human motion from unreliable demonstrations and use it as a rough clue for solving difficult planning problems instead of as a strict solution. During the process of ERTC-HD, robot motions generated from extracted comprehensive features of human motion are saved as a path experience and modified through the path adaptation process of the existing ERTC planner when transferring it to the new problem. The results of three simulation experiments revealed that the ERTC-HD could identify valid motion in cluttered environments within a shorter time than other state-of-the-art planners, even when using unreliable demonstration data collected by a single RGB camera. Reducing the required accuracy of the original information resources can extend the range of LfD framework applications.