Towards Transferring Grasping from Human to Robot with RGBD Hand Detection

Abstract

The task of transferring human knowledge and capabilities to robots is still an open problem. In this paper, we address the problem of transferring human grasping locations of a particular object to a robot manipulator. Using an RGBD sensor, we propose a computer vision based method for human hand detection. This method implements a pixelwise hand detection method with the Random Forest classification algorithm in the color channel. It also creates a kernel-based hand detection method in the depth channel. Based on the theory of joint probability, it fuses both color and depth cues. As a result, this method is able to deal with noisy background and occlusion. Moreover, we apply this method to a grasping task example. In our test, the robot is able to gain the grasping knowledge from visual observation. Our method is complemented with experimental results on the settings of four different sequences with different level of difficulties, and has achieved high performance with respect to hand detection accuracy in comparison with RGB and Depth only methods.