Robotic Learning of Manipulation Tasks from Visual Perception Using a Kinect Sensor

Abstract

The paper addresses the problem of transferring new skills to robots from observation of human demonstrated skill examples. An approach is presented for retrieving trajectories of an object, being manipulated during the demonstrations, from Kinect-provided measurements. The problem of object tracking across the image frames is solved by using weighted dynamic template matching with normalized cross-correlation. Such approach takes advantage of the simultaneous image and depth measurements by the Kinect device in leveraging the pattern localization and pose estimation. Demonstrated trajectories are stochastically encoded with hidden Markov model, and the obtained model is exploited for generation of a generalized trajectory for task reproduction. The developed methodology is experimentally validated in a real-world task learning scenario.