Dynamic Disassembly Planning of End-of-Life Products for Human–Robot Collaboration Enabled by Multi-Agent Deep Reinforcement Learning

Abstract

Disassembly is a critical step in the remanufacturing of end-of-life products. High labor costs and the limited ability of robots to perform intricate disassembly tasks have led to the increasing use of human‒robot collaboration (HRC) for disassembly. This paper addresses a challenge in HRC-based disassembly, i.e., the inherent human uncertainty during disassembly. The uncertainty is that the disassembly time for a task and the task sequence selection by a human during execution might differ from the pre-defined disassembly plan so that dynamic disassembly planning for subsequent tasks is necessary. Stackelberg equilibrium-enabled disassembly task assignment policies are designed to meet the above purpose efficiently and safely. The human leader's policy is to choose tasks that maximize the efficiency-related return value based on the robot's optimal response to the human's choice. As the follower, the robot selects tasks that maximize the safety-related return value for each human task choice. To identify the optimal values of the policies to ensure the safety and efficiency of the entire HRC-based disassembly process, an improved multi-agent proximal policy optimization (i-MAPPO) algorithm is designed. Finally, a case study for disassembling an electric vehicle battery is used to verify that the proposed approach can adapt to human uncertainty with a high success rate while ensuring that the disassembly time remains short and the human-robot distance remains within the safety threshold throughout the disassembly process.