Optical flow-based segmentation of moving objects for mobile robot navigation using pre-trained deep learning models

Abstract

The segmentation of moving objects from video sequences is an intensely researched topic. The two leading application domains are video surveillance systems and self-driving vehicles. Many of the current approaches use some deep learning architecture to accomplish the task. For the training of such models, an extensive collection of input data is necessary. However, it is hard to find an openly accessible, properly annotated dataset for this task in the field of mobile robotics, so one has to create their own. Deep learning models of high complexity easily overfit small datasets if they are trained from scratch, therefore readily available pre-trained networks should be better used. The pretrained deep learning models usually expect RGB images as their input, meanwhile most of the state-of-the-art approaches for moving object segmentation incorporate optical flow data in the prediction process as well. In this paper, we propose a moving object segmentation deep neural network for avoiding dynamic obstacles in the case of indoor mobile robot navigation. The network was built on the top of a deep learning model that was pre-trained for RGB image recognition. We encode the optical flow data and the video frames together into a format that can be processed by the pre-trained network. We introduce a compound loss and training strategy for the model. The empirical evaluation results show that our network predicts satisfactory segmentation masks on the test data for the task of dynamic obstacle avoidance in indoor mobile robot navigation while operating in real-time ($\sim 30$ fps) and being robust to camera movement and the appearance of moving objects.