Embodied Generative AI Art for Enhanced Human-Robot Interaction Through a Human-Centric LLM-Guided Robotic Arm Drawing System

摘要

Generative AI is transforming the way humans interact with robots by integrating language-driven comprehension with embodied execution. While recent research leveraging large language models (LLMs) to enhance communication between humans and machines has shown significant progress, the exploration of integrating LLMs with robots to assist users in creating real-world artistic works remains challenging. This research introduces a novel GENAI-driven intelligent robotic arm drawing system. We fine-tuned GPT-3.5 Turbo to better capture meaningful information from conversations with users and output precise drawing commands, which are directly fed into the fine-tuned Stable Diffusion model to generate desired images. A UFactory xArm equipped with an end-effector holding a drawing pen is deployed and controlled by a speed control algorithm to perform accurate drawing movements based on AI-generated images. Our proposed framework facilitates enhanced HRI experiences with a focus on drawing tasks, effectively embodying the capabilities of generative AI in the field of artistic creation and enabling users to engage deeply in the process of using AI for art creation in real-world environments.