Large Language Model for Humanoid Cognition in Proactive Human-Robot Collaboration

Abstract

Proactive Human-Robot Collaboration (HRC), which aims to achieve mutual-cognitive, predictable, and self-organizing collaboration between multiple humans and robots, is crucial for today’s human-centric smart manufacturing. To enable Proactive HRC, various methods have been explored, including deep neural networks for visual detection, scene graph for decision-making, and reinforcement learning for robot execution. However, these methods often require re-training with domain-specific datasets in different scenarios, lacking generalizability and transferability for diverse manufacturing activities. The advent of Large Language Model (LLM) technology offers a promising solution for comprehending diverse tasks, modelling human intentions, and planning robot operations using natural vision-language instructions. This ability closely resembles human intelligence, specifically humanoid cognition, which allows flexible knowledge acquisition of the surrounding environment and exerting physical influence on tasks. Therefore, this paper delves into the concept of humanoid cognition in Proactive HRC and evaluates relevant LLM methods from the perspectives of task explainability, human-centricity, and robot executability. Based on the testing results, the authors provide discussions and future prospects for successfully integrating LLM approaches into Proactive HRC in the manufacturing domain.