Automated Vision-based Bolt Sorting by Manipulator for Industrial Applications

摘要

In industrial automation, nuts and bolts are key components that facilitate assembly processes. Estimating their sizes and sorting them accordingly would be a significant advancement for automation in industrial settings. This paper introduces a distinct approach by amalgamating deep learning based segmentation with classical vision for automated bolt sorting in unstructured industrial environment. We evaluated advanced deep learning algorithms PPLiteSeg(STDC I&II), OCRNet, and DeepLabV3+ on our custom bolt dataset. DeepLabV3+ achieved the best performance over the other models in accurate bolt segmentation, with a remarkable mean Intersection over Union (mIoU) of 0.95. Custom bolt dataset is prepared considering various environmental conditions and included both new and aged bolts. We present a comparative analysis of the segmentation results for each learning model mentioned. To estimate bolt size, we integrated the segmented binary image with depth information from Intel RealSense D415 camera. Through a series of experiments, we successfully localized bolts (both position and orientation) relative to robot’s base and sorted them according to their size by using Kinova Gen3 Lite robotic arm. The proposed approach has the potential to significantly advance the automation for assembly lines and warehouse management within industries, particularly when dealing with small objects.