Recognizing and Interpreting Gestures within the Context of an Intelligent Robot Control Architecture

Abstract

In this paper we describe a stereo vision system that can recognize natural gestures and an intelligent robot control architecture that can interpret these gestures in the context of a task. The intelligent control archi-tecture consists of three layers. The top layer delib-erates via a state-based, hierarchical, non-linear plan-ner. The bottom layer consists of a suite of reactive skills that can be configured into synchronous state machines by the middle layer. The middle layer, im-plemented in the RAPs system, serves to mediate be-tween the long range deliberation of the top layer and the continuous activity of the bottom layer. It is in the middle layer that we are investigating context focused deictic gesturing for human-robot interaction. When directed in the context of different RAPs, human ges-tures can be interpreted by the system differently for different tasks. This work shows that a) our archi-tecture, designed to support perception and action, can also support other forms of communication, and b) task contexts can act as resources for communica-tion by simplifying the interpretation of communica-tive acts.