RGBD Fusion Grasp Network with Large-Scale Tableware Grasp Dataset

Abstract

This paper proposes a novel approach to address the technical challenges of stable object grasping, particularly in the context of handling tableware in a home environment. Handling tableware is particularly important, yet challenging, due to the flat nature of most tableware objects and the need to maintain a stable posture to prevent spills. To address these challenges, we present three key contributions: 1) a large-scale tableware dataset, not commonly found in the previous datasets; 2) a novel sampling method for stable grasp pose generation; and 3) a multi-modal fusion grasp network that effectively learns 6- DoF grasp pose, including flat objects. Our dataset contains over 45 million grasp poses and 1 million RGBD images captured in 800 scenes, which include randomly selected 10–18 tableware objects under 4 different lighting conditions. The grasp poses in the dataset are generated using a novel sampling method that incorporates geometric analysis to ensure stable grasping with minimal object movement. Furthermore, we design an RGBD fusion grasp network (RGBD-FGN) that can combine information from RGB and depth images considering each characteristic. Our experimental results demonstrate the superior performance of our approach over existing techniques, which is a significant contribution towards developing a multitasking home robot. Our dataset and source code can be accessed at https://github.com/SamsungLabs/RGBD-FGN.