MIS-SLAM: Real-Time Large-Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing

Abstract

Real-time simultaneous localization and dense mapping is very helpful for providing virtual reality and augmented reality for surgeons or even surgical robots. In this letter, we propose MIS-SLAM: A complete real-time large-scale dense deformable SLAM system with stereoscope in minimal invasive surgery (MIS) based on heterogeneous computing by making full use of CPU and GPU. Idled CPU is used to perform ORB-SLAM for providing robust global pose. Strategies are taken to integrate modules from CPU and GPU. We solved the key problem raised in the previous work, that is, fast movement of scope and blurry images make the scope tracking fail. Benefiting from improved localization, MIS-SLAM can achieve large-scale scope localizing and dense mapping in real time. It transforms and deforms current model and incrementally fuses new observation while keeping vivid texture. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">In-vivo</i> experiments conducted on publicly available datasets presented in the form of videos demonstrate the feasibility and practicality of MIS-SLAM for potential clinical purpose.