OperateLLM: Integrating Robot Operating System (ROS) Tools in Large Language Models

摘要

Rapid advancements in the field of large language models (LLMs) have demonstrated extensive use in various domains, including integration with domain-specific tools, but their application in assisting the development of robotic systems is still underexplored. We present OperateLLM, an integration of ROS 2 (Robot Operating System 2) tools with LLMs, that is designed to enhance the convenience and speed of robotic development. We leverage DeepSeek Coder v2 as the core LLM and uses the Python API for ROS 2, rclpy. OperateLLM has shown the ability to create fundamental ROS 2 components such as Nodes and Publishers and perform other basic tasks, as well as perform more complex tasks with the ROS 2 API. By integrating LLMs with specialized tools for the ROS, OperateLLM bridges the gap between high-level language understanding shown by LLMs and low-level robotic control often required by humans. Although OperateLLM requires a human operator to control it, the system increases the ease of using the ROS 2 API significantly, facilitating more efficient and intuitive interactions. Evaluation of OperateLLM through a series of robotics tasks that require the use of ROS 2 demonstrates its proficiency in streamlining operations that usually require human intervention to a significant extent. This integration not only enhances advanced robotics research but also makes robotic development more accessible by lowering the technical barrier for non-experts, helping to promote innovation and practical advancements in robotics. Our model outperforms GPT-4 on a number of measurements, including the mean scores experts assigned to reasoning provided by both models and a 90% accuracy rate versus GPT-4's 60% accuracy rate on a set of advanced prompts <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(n=10)$</tex>. We also significantly focus on the cybersecurity implications for such a model, including the potential for such a system to be attacked via adversarial prompting and how data can potentially be leaked from sensitive robotic systems. We propose a number of solutions to this, including potential levels of access control in organizations. In order to combat adversarial prompts, we propose the potential use of self-reminders, which have been shown to be effective in contending with adversarial prompting.