Neuro-Control of a Robot Using Visual and Force/Torque Sensor Information (Manipulation of a Flexible Beam Object)

Abstract

In order for a robot to manipulate a flexible beam object, methods of establishing a deformation model of the beam object and analysis of the contact force detected by a force/torque sensor have been widely introduced. However, those methods can not be applied when the physical properties of the beam object are unknown. In this paper, sensor fusion of visual and force/torque information using on-line learning neural networks is proposed. The proposed method is applied to a 6 DOF robot to perform a beam-in-hole task. To detect deformation of the beam object, two CCD cameras are used and real time image processing has been developed. Visual information obtained from the two CCD cameras and force/torque information obtained from a force/torque sensor are integrated by three multi-layer neural networks. The outputs of the neural networks are connected to a proposed controller to generate the desired velocity of the robot. Experiments using different beam objects have been made successfully demonstrating the effectiveness of the proposed method.