Fusion Of Vision And Touch For Spatio-Temporal Reasoning In Learning Manipulation Tasks

摘要

This paper presents a framework for the fusion of vision and touch, useful in learning various manipulation tasks by a robot arm. Initially the robot has poor knowledge of the laws that govern the behavior of objects, and incomplete knowledge about physical features of individual objects. We analyse the fusion of vision and touch for learning object manipulation tasks, various methods of features acquisition, and an architecture of the system that provides feedback between sensing, manipulating and learning. Simple control loops allow the system to execute the manipulation tasks and to learn such a selection of the values of control parameters that prevents faults and object damage. The main emphasis is on learning. In sections 5 and 6 we demonstrate how the system discovers new regularities, how it recognizes new and useful object properties, and how the performance on similar tasks can be improved by application of newly acquired knowledge. Sections 1-4 describe a preliminary design of an architecture that allows for application of sensor fusion and for learning by improving manipulation skills by a robot arm.