FFBGNet: Full-Flow Bidirectional Feature Fusion Grasp Detection Network Based on Hybrid Architecture

Abstract

Effectively integrating the complementary information from RGB-D images presents a significant challenge in robotic grasping. In this letter, we propose a full-flow bidirectional feature fusion grasp detection network (FFBGNet) based on a hybrid architecture to generate accurate grasp poses from RGB-D images. First, we construct an efficient Cross-Modal Feature fusion module as a bridge for information interaction in the full flow of the two branches, where fusion is applied to each encoding and decoding layer. Then, the two branches can fully leverage the appearance information in the RGB images and the geometry information from the depth images. Second, a hybrid architecture module for CNNs and Transformer parallel is developed to achieve better local feature and global information representations. Finally, we conduct qualitative and quantitative comparative experiments on the Cornell and Jacquard datasets, achieving grasping detection accuracies of 99.2<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\%}$</tex-math></inline-formula> and 96.5<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\%}$</tex-math></inline-formula>, respectively. Simultaneously, in physical grasping experiments, the FFBGNet achieves a 96.7<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\%}$</tex-math></inline-formula> success rate in cluttered scenes, which further demonstrates the reliability of the proposed method.