A Review of Critical Deep Learning-Based Image Guidance Technologies for Surgical Robots

Abstract

Despite significant advancements in the medical field, surgical robots still exhibit limited autonomy. Image guidance techniques are widely regarded as a key pathway toward achieving full automation in surgical robotics, and deep learning (DL) has further accelerated progress in this domain. However, to our knowledge, no existing study has systematically reviewed the key technologies involved. Therefore, this article provides a comprehensive analysis of the critical technologies underlying DL-based image guidance for surgical robots. First, we introduce medical image classification methods and commonly used datasets. The discussion then examines five preoperative technologies in image guidance, including image segmentation, registration and fusion, 3-D reconstruction, and surgical planning, with emphasis on their current research status and associated challenges. Next, we review four intraoperative critical technologies, including real-time registration and planning, surgical instrument tracking, and soft-tissue detection, while summarizing the challenges and limitations of DL applications in this domain. We also explore the potential applications of advanced technologies in image-guided surgical robotics, such as extended reality and foundation models. Furthermore, by consolidating preoperative, intraoperative, and advanced critical technologies, we propose a five-level framework for evaluating image-guidance capabilities. Finally, we outline research trends and future directions.