Deep Learning for Manipulator Visual Positioning

Abstract

To solve the problem of robot visual servoing on unknown object, we apply deep learning to the robotics control and propose a manipulator visual positioning method based on deep learning. Firstly, we use a well-trained object detection network based on deep learning to recognize and detect objects in the first frame image, which is captured by the camera mounted on the robot; Then, user randomly selects object to be manipulated through human-computer interaction according to the result of recognition and detection. In the sequence of subsequent images, robotics control system can detect the object to be manipulated using this object class label information and well-trained object detection network; Finally, we can calculate the features error and design the visual sliding mode controller to control the manipulator to position the unknown object randomly selected by the user in the first frame image. We conduct two robotics experiments of visual positioning on unknown object in the complex nature scenes using a MOTOMAN-SV3X industrial manipulator. Experimental results show that our manipulator visual positioning method can position an unknown object without having to known any model information about the object prior to the positioning task execution, and it is an effective manipulator visual positioning on unknown object method.