Sim2Real Rope Cutting With a Surgical Robot Using Vision-Based Reinforcement Learning

Abstract

Cutting is a challenging area in the field of autonomous robotics but is especially interesting for applications such as surgery. One large challenge is the lack of simulations for cutting with surgical robots that can transfer to the real robot. In this work, we create a surgical robotic simulation of rope cutting with realistic visual and physics behavior using the da Vinci Research Kit (dVRK). We learn a cutting policy purely from simulation and sim2real transfer our learned models to real experiments by leveraging Domain Randomization. We find that cutting with surgical instruments such as the EndoWrist Round Tip Scissors comes with certain challenges such as deformations, cutting forces along the jaw, fine positioning, and tool occlusion. We overcome these challenges by designing a reward function that promotes successful cutting behavior through fine positioning of the jaws directly from image inputs. Policies are transferred using a custom sim2real pipeline based on a modular teleoperation framework for identical execution in simulation and the real robot. We achieve a 97.5% success rate in real cutting experiments with our 2D model and a 90% success rate in 3D after the sim2real transfer of our model. We showcase the need for Domain Randomization and a specialized reward function to achieve successful cutting behavior across different material conditions through optimal fine positioning. Our experiments cover varying rope thicknesses and tension levels and show that our final policy can successfully cut the rope across different scenarios by learning entirely from our simulation. Further project information is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://medcvr.utm.utoronto.ca/tase2024-cutrope.html</uri>Note to Practitioners—Cutting is one of many repetitive tasks during a surgical procedure that can be automated to reduce a surgeon’s fatigue. This paper presents an autonomous approach to cutting that can be learned from simulation. The approach builds a surgical robotic rope environment in simulation that mimics the visuals and physics of real-life rope cutting. An autonomous agent, trained with learning-based methods, uses images to position the surgical scissors and controls the jaw to perform a cut on the rope. The agent creates quality cuts by learning to finely position the jaws optimally near the scissor joint, which is learned through rewards in simulation. Agents from the simulation are transferred to the real robot setup using a custom modular ROS teleoperation framework, treating deep learning-based autonomous agents as input into a teleoperation scheme. Experiments were conducted to examine the success of the method with different material conditions and the effect of fine positioning on cut success.