Attention-based Deep Learning for Visual Servoing

Abstract

The traditional image based visual servo(IBVS) system relies on manually extracted features, the process of estimating the feature Jacobian is very complicated and difficult. This paper proposes an approach of visual servo control which has an end-to-end structure. We find the excellent performance of convolutional neural network in classification and regression tasks and make use of it, attention mechanism is introduced and region of interest(ROI) is extracted through the feature extraction to strengthen the expression of feature information. The corresponding relationship between image space and pose space is successfully obtained. The dataset is collected by performing perspective transformation on the image. The proposed dual-stream network processes the input images which representing the current task pose and the desired task pose at the same time, and The command obtained from the network output can control the robot to complete the corresponding visual servo task. The proposed approach has achieved good results in the corresponding experimental scenes.