Automated Vision-Based Bolt Handling for Industrial Applications Using a Manipulator

Abstract

Fasteners are critical components in industrial assembly processes. Accurate identification and size estimation of fasteners are essential for enhancing the efficiency and precision of these processes. However, the detection and segmentation of fasteners in factory environments are challenging due to their limited visual features and frequent overlap. This paper presents a novel approach for automated bolt handling using 3D point cloud data acquired from an RGB-D camera. Our method employs k-means clustering for segmentation and Oriented Chamfer matching for precise pose estimation, enabling a 6-DOF robotic arm to accurately pick and place bolts. Color-based segmentation and contour analysis facilitate size estimation for subsequent sorting. Experiments conducted in a simulated shop floor environment using Gazebo (ROS1) with a Kinova Gen3 Lite arm demonstrated the system’s ability to handle overlapping bolts and achieved bolt sorting without any miss-classifications. This research contributes to enhanced automation and productivity in industrial automation, with potential applications in various industries requiring precise component handling, such as automotive, aerospace, and electronics.