ENeRF-SLAM:#A Dense Endoscopic SLAM With Neural Implicit Representation

Abstract

Quantitative calculations of camera poses and dense anatomical reconstructions from endoscopic videos are essential for applications such as intraoperative navigation and robotic surgery automation. Prior studies on this task either overlook the unique characteristics of endoscopic scenes or produce reconstructions with numerous gaps due to limited observations, significantly limiting their practical application. Inspired by recent advancements in neural rendering, we develop a dense visual SLAM system that employs neural implicit representations, specifically designed for endoscopic sequences. By incorporating 3D geometric scene priors, our system effectively predicts and fills in unseen areas, ensuring the continuous and complete reconstruction of the scene. Taking into account the dynamic nature of the light source and the confined anatomy of the human body, we propose a neural implicit representation method designed for endoscopic scenes. Additionally, we introduce a hybrid tracking method that merges Gauss-Newton and gradient-based pose optimization, improving geometric consistency across frames. This reduces reliance on precise data matching and significantly enhances camera tracking accuracy. Extensive experiments on both synthetic and real medical endoscopic datasets demonstrate that our method outperforms existing systems in terms of scene reconstruction quality, camera tracking accuracy, and image rendering quality. Code is available at: https://github.com/Mar-lll/ENeRF-SLAM.