Artificial intelligence based real-time segmentation and feature tracking in urological retroperitoneal robotic assisted surgery

Abstract

BACKGROUND: Robotic-assisted surgery, particularly with the da Vinci Single-Port system, enables precise operations in confined anatomical spaces like the retroperitoneum. However, the complexity of retroperitoneal anatomy challenges spatial orientation, especially for novice surgeons. This study aimed to develop and validate an Artificial Intelligence (AI)-based framework for real-time segmentation and feature tracking in Single-Port retroperitoneal robotic-assisted nephrectomies. METHODS: A convolutional neural network based on YOLACT++ was used for instance segmentation. The model was trained on annotated video frames from 15 patients who underwent SP nephrectomies. Key anatomical landmarks and surgical instruments were segmented and detected in real-time. Model performance was evaluated using the Area Under the Precision-Recall Curve. RESULTS: The model achieved Area Under the Precision-Recall Curve values ranging from 0.759 to 0.901 for anatomical landmarks and up to 0.887 for surgical instruments. Limitations include a small dataset (150 annotated frames), which may lead to overfitting, and lower segmentation precision for some features, such as the ureter. Further validation in live surgical settings and diverse procedures is needed. CONCLUSIONS: This study demonstrates the potential of AI for real-time surgical guidance in Single-Port retroperitoneal procedures. The proposed framework could enhance spatial orientation, improve training for novice surgeons, and streamline complex retroperitoneal surgeries. Future research will focus on expanding the dataset, improving segmentation accuracy, and integrating the model into live surgeries for broader applicability.