Associating Grasping with Convolutional Neural Network Features.

摘要

In this work, we provide a solution for pre-shaping a human-like robot hand for grasping based on visual information. Our approach uses convolutional neural networks (CNNs) to define a mapping between images and grasps. Applying CNNs to robotics applications is non-trivial for two reasons. First, collecting enough robot data to train a CNN at the same scale as the models trained in the vision community is extremely difficult. In this work, we demonstrate that by using a pre-trained CNN, a small set of grasping examples is sufficient for generalizing across different objects of similar shapes. Second, the final output of a CNN contains little location information of the observed object, which is essential for the robot to manipulate the object. We take advantage of the hierarchical nature of CNN layers and identify the 3D positions of features that capture the hierarchical support relations between filters in different CNN layers using an approach we call targeted backpropagation. Targeted backpropagation traces the activation of higher level features in a CNN backwards through the network to discover the locations in the observation that were responsible for making them fire, thus localizing important structures that are manipulable in the environment. We show that this approach outperforms approaches without targeted backpropagation in a cluttered scene. We further implement a hierarchical controller that controls fingers and palms based on features located in different layers of the CNN for pre-shaping the robot hand and demonstrate that this approach outperforms a point cloud based approach on a grasping task on Robonaut-2.