Workpiece Segmentation Based on Improved YOLOv5 and SAM

Abstract

In the manufacturing industry, especially in customized assembly lines, robots often need to grasp disorderly stacked and mixed workpieces. For the robot arm to effectively complete this task, it is necessary to obtain the pose information of the workpiece. Using a regular RGB camera, the workpiece to be captured is first segmented from the scene image; the segmented part is then used to obtain the pose information of the workpiece. In this paper, only the segmentation work is involved. Traditional image segmentation algorithms rely on human-designed features, which are often shallow and limited to specific image properties such as color and threshold. This results in suboptimal segmentation performance. Deep neural networks overcome this limitation by learning deep features from images that are more informative and discriminative; therefore, achieve more accurate segmentation results. To improve the detection performance of disordered mixed workpieces, YOLOv5 is enhanced with an attention mechanism to extract more features. The detection results are then transferred to the SAM model as prior information to segment each workpiece. Using binocular vision, we can obtain the depth information of each workpiece. We can then select the topmost workpiece with the smallest depth as the target to be captured and segment it from the scene image. Experimental results show that the improved YOLOv5 algorithm has an average accuracy of 1.6% higher than the original algorithm. Combined with the SAM model, the workpieces to be captured are segmented, and the segmentation accuracy is 2.6% higher than the original YOLOv5 and SAM models.