Pointing gesture-based unknown object extraction for learning objects with robot

Abstract

This paper describes how to extract a unknown object pointed by a person while interacting with a robot. Our proposed method consists of two stages: the detection of the operators’ face and the estimation of the pointing direction, extraction of the pointed object. The operator’s face is recognized by using the Haar-like features. Then, from the shoulder-to-hand line we estimate the pointing direction. Finally, we segment an unknown object from 3D point clouds in region of interest. We implemented a object registration system with our mobile robot and obtained reliable experimental results.