Learning issues in a multi-modal robot-instruction scenario

摘要

Abstract — One of the challenges for the realization of future intelligent robots is to design architectures which make user instruction of work tasks by interactive demonstration effective and convenient. A key prerequisite for enhancement of robot learning beyond the level of low-level skill acquisition is situated multi-modal communication. Currently, most existing robot platforms still have to advance to make the development of an integrated learning architecture feasible. We report on the status of the Bielefeld GRAVIS-robot architecture that combines statistical methods, neural networks, and finite state machines into an integrated system for instructing grasping tasks by human-machine interaction. It combines visual attention and gestural instruction with an intelligent interface for speech recognition and linguistic interpretation and a modality fusion module to allow multi-modal task-oriented communication. It further integrates imitation of human hand postures to allow flexible grasping of every-day objects. With respect to this platform, we sketch the concept of a learning architecture based on several interlocking levels with the goal to demonstrate speech-supported imitation learning of grasping. I.