Extraction of fine motion through multiple observations of human demonstration by DP matching and combined template matching

Abstract

The paper describes research on how a robot, through observation of human demonstrations, can learn task level representations of human hand-work tasks. We propose a technique for segmenting an observed hand-work task into pieces which are composed of fine motion or coarse motion. Fine motion means delicate manipulation and holds the relative trajectory between the grasped object and the target object, while coarse motion is a symbol which connects each fine motion. During coarse motion, a trajectory can be adjusted according to the environment or the structure of a robot when the robot performs the same task performed by a human. To extract essential fine motion automatically, we propose a technique for integrating and aligning multiple observations of different demonstrations, which are virtually the same task, by using data gloves and multi-dimensional dynamic programming (DP) matching. Along each fine motion, the relative trajectory is calculated by tracking the manipulated object using stereo vision. We propose a model-based localization technique which combines 2D and 3D template matching. We have implemented those techniques on our human-form robot and present an experimental result which analyzed and performed a noncontact hand-work task.