IGN: Instance-Guided Net for 3-D Instance Segmentation in Cluttered Scenes via Monocular Depth Sensor

Abstract

The 3-D instance segmentation is a fundamental task in sensor data processing for robot manipulation. However, recent works in this field did not pay much attention to 3-D instance segmentation under cluttered conditions. Specifically, current widely-used 3-D instance segmentation datasets mainly focus on indoor scenes, and they often reconstruct the whole 3-D scene by multiple depth sensors. Therefore, there are no occluded or cluttered areas in these datasets. Moreover, we explore prevailing 3-D instance segmentation models and find that the traditional heuristic algorithms in these models severely harm their performance in cluttered scenes. To tackle these issues, we first build a monocular depth sensor system to collect a new dataset, named cluttered objects (CO), which is tailored for robotic manipulation under cluttered conditions. Then, we propose instance-guided net (IGN) to segment targets in these complex areas. In our IGN, the pointwise instance information is fully exploited to make all heuristic algorithms task-oriented. The keys to IGN are dual-instance-guided block (D-IGB), instance-guided upsampler (IGUS), and instance-guided downsampler (IGDS). In D-IGB, we build the instance-aware receptive field and dilated receptive field to gather intrainstance information and long-range context simultaneously. In IGUS, the instance-aware receptive field is applied to realize task-oriented interpolation. Moreover, we propose IGDS to retain more beneficial foreground features by eliminating background points. Extensive experiments on CO and other public datasets show the effectiveness of our IGN.