Real-Time Interactive Capabilities of Dual-Arm Systems for Humanoid Robots in Unstructured Environments

Abstract

The efficient and rapid processing of target positions is crucial for the real-time interactive motion operations of humanoid robots within unstructured acquisition. This paper presents a novel strategy for the acquisition and transfer of target positions, optimizing the coordination of dual-arm systems in humanoid robots, essential for improving grasping speed and environment interaction responsiveness. The proposed approach integrates a camera mounted on the observer arm for real-time target detection, alongside an executor arm responsible for performing interactive tasks. The mechanism performs seamless and swift target acquisition of positions, converting them from the observer to the executor arm’s operational frame. We utilize YOLOv8 for instant target object detection in RGB images, coupled with depth image analysis for accurate object interaction. A calibration method, enhanced by a plus sign median filter (PSMF), is introduced to improve the accuracy and precision of the depth data. Additionally, a novel kinematic technique is proposed to expedite the position transfer process. Simulation and experimental validations show the efficiency of our kinematic approach, which is demonstrated to be nearly 50 times faster than traditional methods. Moreover, the PSMF calibration technique significantly elevates the robot’s grasping response and success rate, showing that the proposed methods augment the interactive capabilities of humanoid robots in dynamic settings.