Learning of demonstrated grasping skills by stereoscopic tracking of human head configuration

Abstract

In this paper a novel approach to learning by demonstration (LbD) is presented. A multimodal service robot is taught grasping skills by a human instructor who demonstrates a grasping action. Our approach contributes novel solutions to the aspects of robustly tracking the demonstrator's hands in real time as well as to the transformation of tracking results into grasping skills. To track the demonstrator's hands in stereoscopic images a mean-shift-like algorithm is adapted. For the very first time this algorithm is applied to local binary patterns (LBP) and color histograms. To retrieve the hand configuration we use view-based principal component analysis (PCA). To develop grasping skills from tracking results the robot repetitively tracks the demonstrator's grasping actions and transforms the results into three-dimensional self organizing maps (SOMs). The SOMs give a spatial description of the collected data and serve as data structures for a reinforcement learning (RL) algorithm which optimizes trajectories for use by the robot. The approach is applied to a multimodal service robot. Experiments show the effectiveness of the LBP-enhanced mean-shift-like tracking and the robustness of LbD based on SOMs and RL