Weld Seam Segmentation in RGB-D Data using Attention-based Hierarchical Feature Fusion

Abstract

Accurate weld detection, segmentation, and positioning based on 3D vision is an important prerequisite for guiding the robots to perform weld milling. It is difficult to accurately segment only by 2D images because multiple welds with different morphological characteristics on the same workpiece may be located in multiple different planes, the lighting conditions sometimes change, and overexposure or underexposure may occur. In response, the method for weld instance segmentation based on the hierarchical feature fusion of RGB-D has been proposed, which can extend the module of RGB-D hierarchical feature fusion based on Mask R-CNN feature pyramid network (FPN). The fusion module introduces channel attention and spatial attention to realize the cross-modality fusion of RGB features and depth features in space. In specific, MaskGIoU Head has been added to the Mask R-CNN model to improve the accuracy of weld seam positioning, and the feature pyramid network (FPN) has been built in Mask Head to improve the boundary quality of the weld segmentation. The method presented in this paper has been proved effective and accurate by the results of the experiment.