Hybrid Tracking Module for Real-Time Tool Tracking for an Autonomous Exoscope

摘要

Exoscopes have emerged as a promising visual solution within the field of microneurosurgery. However, manual repositioning poses a challenge causing interruptions that disrupt the surgical flow. Thus, the need for hands-free exoscope control arises. This paper introduces a position-based visual-servoing control approach, comprising a detection module, a hybrid tracking module, and a control module that adjusts a robotic camera holder to follow a surgical tool. The hybrid module was integrated to track and predict the surgical tool's future position to minimize system latency. The proposed system is composed of a 7 Degree-of-Freedom robotic manipulator with an eye-in-hand stereo camera. A comparative analysis with three alternative approaches (Convolutional Neural Network - CNN, Particle Filter - PF, Optical Flow - OF) was assessed using Tracking Error and Center Error metrics. Results showed improved tracking accuracy with an average error of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$9.84 \pm 0.08$</tex-math></inline-formula> mm for slow movements (2.5 cm/s) and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$13.11 \pm 0.39$</tex-math></inline-formula> mm for rapid movements (4 cm/s). Finally, a User Study was conducted to investigate whether the proposed system effectively reduced the users' workload compared to the manual repositioning of the camera.