Robotic Visual Servoing Based on Convolutional Neural Network

Abstract

In order to simplify the problems of image feature choosing and extraction, and the nonlinear mapping estimation in traditional visual servoing, we present a visual servoing method based on convolutional neural network (CNN) to realize the precise, robust, and real-time 6DOF control of robotic manipulation. Herein we propose an approach to design and generate a dataset from a single image, by simulating the motion of eye-in-hand robotic system. This dataset is utilized to train our visual servoing CNN which computes the relative pose between robotic system and external environment. Then the output of the network is employed in the visual control schemes. This method converges robustly with experimental result of a position error less than one millimeter and a rotation error less than half a degree in average in simulation.