Mask-Grasp R-CNN: Simultaneous Instance Segmentation and Robotic Grasp Detection

Abstract

Autonomous robotics research has been driven by rapid advancements in deep learning architectures and the ability to use transfer learning to train networks using smaller datasets. This paper proposes a single deep convolutional neural network capable of simultaneously predicting objects in a scene, their segmentation mask, and a ranked list of the optimal grasping locations. For the first time in grasp detection, adaptive-size anchors are proposed as prior information for training. The proposed approach, named Mask-Grasp R-CNN, shows that an object detection and instance segmentation network can be easily extended for the grasp detection task without modifying any of its weights. Building on a Mask R-CNN network, the proposed approach detects grasping points at an instance level rather than at the image level. This enables Mask-Grasp R-CNN to achieve a 10% reduction in miss rate at 1 false-positive-per-image when evaluated on the Multi-Object dataset. The end goal is to integrate this system into a semi-autonomous control scheme to be used in upper-limb prosthetics.