Estimating future forceps movement using deep learning for robotic camera control in laparoscopic surgery

Abstract

In laparoscopic surgery, an assistant surgeon must hold the laparoscope. Autonomous control of the holder helps the surgeon focus on the surgery. Estimating the future movement of the forceps can improve the control performance of a holder robot. We have previously proposed a method for estimating the position of the forceps 0.1 seconds ahead in a 2D image by using deep learning, based on segmented forceps in the camera image. In this study, we extend the prediction time to 0.1-3 seconds ahead, and investigate the accuracy of the estimation when the number of past position inputs to the convolutional neural network changes. We confirm that the forceps position in a 2D image 0.8 seconds ahead can be estimated online using only the past three positions in the suturing task within an error of 30 pixels, which is acceptable for laparoscope holder control.