A Digital Twin Framework for Robot Grasp and Motion Generation

Abstract

Digital twin technology has demonstrated significant potential across various domains. In the field of robotics, manipulation, particularly robotic grasping, is fundamental to many applications. While artificial intelligence and deep learning methods have been employed to enhance grasp detection through visual perception, challenges such as data collection, the simulation-toreality (sim2real) gap, and motion planning safety still persist. To address these challenges, this paper proposes a comprehensive framework for digital twin-based robotic grasping and motion generation. Firstly, a digital twin synthetic data generation and domain randomization framework for robot manipulation is proposed to generate large-scale data to support deep learning training. Then, a hybrid training strategy combining synthetic and real data is proposed to reduce sim2real gap and enhance model performance in real-world perception. Finally, a dynamic digital twin working environment construction framework is proposed for motion generation and collision avoidance in changing environments. The proposed framework is widely applicable to various robotic tasks, offering a robust solution to the challenges faced in robotic manipulation. By leveraging digital twins, the efficiency, safety, and reliability of robotic operations will be enhanced across multiple robotic applications.