A Modified Q-Learning Algorithm for Robot Path Planning in a Digital Twin Assembly System

摘要

Abstract Product assembly is an important stage in complex product manufacturing. How to intelligently plan the assembly process based on dynamic product and environment information has become an pressing issue needs to be addressed. For this reason, this research has constructed a digital twin assembly system, including virtual and real interactive feedback, data fusion analysis and decision-making iterative optimization modules. In the virtual space, a modified Q-learning algorithm is proposed to solve the path planning problem in product assembly. The proposed algorithm speeds up the convergence speed by adding dynamic reward function, optimizes the initial Q table by introducing knowledge and experience through the case-based reasoning (CBR) algorithm, and prevents entry into the trapped area through the obstacle avoiding method. Finally, take the six-joint robot UR10 as an example to verify the performance of the algorithm in the three-dimensional pathfinding space. The experimental results show that the modified Q-learning algorithm's pathfinding performance is significantly better than the original Q-learning algorithm.