Real-time Detection and Tracking of Surgical Instrument Based on YOLOv5 and DeepSORT^*

摘要

To enable the operator to control the surgical assistant robot without a separate joystick, we aim to develop an interface using artificial intelligence that responds to the movement of surgical instruments in the laparoscopic screen. In this study, we propose a method for detecting and tracking surgical tools using the YOLOv5 (YOU ONLY LOOK ONCE v5) and DeepSORT (Deep Learning based Simple Online and Real-time Tracking) algorithms. The proposed approach employs the YOLOv5 algorithm to detect surgical tools in real-time, and the DeepSORT algorithm to track the detected tools across multiple frames. The YOLOv5 model requires minimal computation, which enables the rapid detection of surgical instruments. Furthermore, the DeepSORT algorithm can precisely track object movements in complex environments. The proposed method’s tracking stability was assessed using the average pixel error performance metric test. To evaluate the method’s performance, we used a 2-degree-of-ffeedom remote center motion experimental setup and employed the PID control algorithm to control the laparoscopic camera’s movements.