Motion Prediction of Beating Heart Using Spatio-Temporal LSTM

Abstract

In robot-assisted cardiac surgery, predicting heart motion can help improve the operation accuracy and safety of surgical robots. Different from the conventional prediction schemes which model the point of interest (POI) with only temporal correlation of past observations, this paper proposes an LSTM-based method by exploiting the spatio-temporal correlation of the 3D movements of POI and auxiliary points (APs) on the same surface of the heart. Three different LSTM models are investigated. The first two models define the POI prediction as a pure time-series forecasting problem based on past POI trajectory, and the third model combines the past observations of POI and new observations of APs to take into consideration the extra spatial correlations for prediction. Experimental comparison studies based on 3D coordinates obtained from real stereo-endoscopic videos demonstrate the superior performance of the proposed spatio-temporal LSTM model.