Segmentation of surgical instruments in laparoscopic videos: training dataset generation and deep-learning-based framework

摘要

Surgical instrument segmentation in laparoscopic image sequences can be utilized for a variety of applications during surgical procedures. Recent studies have shown that deep learning-based methods produce competitive results in surgical instrument segmentation. Difficulties, however, lie in the limited number of training datasets involving surgical instruments in laparoscopic image frames. Even though there are publicly available pixelwise training datasets along with trained models from the Robotic Instrument Segmentation challenge, we are not able to relate them to laparoscopic image frames from different surgical scenarios without any pre- or postprocessing. This is because they contain different instrument shapes, image backgrounds, and specular reflections, which implies laborious manual segmentation for training dataset generation. In this work, we propose a novel framework for semi-automated training dataset generation for the purpose of robust segmentation using deep learning. To generate training datasets in various surgical scenarios faster and more accurately, we utilize the publicly available trained model from the Robotic Instrument Segmentation challenge and then use the Watershed Segmentation-based method. For robust segmentation, we use a two-step approach: first, we obtain a coarse segmentation obtained from a deep convolutional neural network architecture, and then we refine the segmentation result via the GrabCut algorithm. Through experiments using four different laparoscopic image sequences, we demonstrate the ability of our proposed framework to provide robust segmentation quality.