Unseen Object Few-Shot Semantic Segmentation for Robotic Grasping

Abstract

Semantic segmentation is a popular technology enabling robots to perceive and interact with the environment sufficiently. However, unseen objects in the new environment make it challenging to obtain accurate segmentation results. To solve this problem, we apply few-shot semantic segmentation in robot perception and propose a complete semantic grasping framework that integrates few-shot semantic segmentation and grasp pose detection for the first time. Our method can quickly identify the unseen target and obtain accurate segmentation results with only a few labeled support images. Meanwhile, our proposed multi-scale prototype extraction module and self-attention-guided feature learning module can generate representative prototypes and query features. Furthermore, we set up a few-shot semantic segmentation benchmark for robotic environments named GraspNet-10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{i}$</tex-math></inline-formula> . Experimental results on the dataset show that our method outperforms previous state-of-the-art methods by a large margin. We also conduct real robotic grasping experiments to demonstrate the feasibility and effectiveness of our pipeline.