Using a 3D World to Address Perceptual Issues in Human-robot Coordination

Abstract

There is a growing body of evidence that human perception is “active”, in the sense that it is largely goal-oriented and top-down. Task goals appear to influence how people perceive their environment. Effective interaction between people and unmanned systems requires that the unmanned systems’ perceptions be comprehensible to people, and this means that the unmanned systems should also perceive the world in an active manner. Recent evidence in cognitive psychology and neuroscience supports the proposition that simulation, the “re-enactment of perceptual, motor and introspective states” is a central cognitive mechanism. Cognitive functions such as anticipation and planning operate through a process of internal simulation of actions and environment. Indeed there is a history in the field of Artificial Intelligence of using “simulated action” as an algorithmic search procedure, e.g., game trees, though such an approach typically has problematic computational complexity. The simulations include not just the effect of actions, but also the understood laws of physics (e.g., will a falling object continue to fall). We are building a robot cognitive architecture that is based on a unified cognitive architecture - Soar - and that uses active perception and simulation in planning. Our system constructs a 3D virtual copy of itself and its environment, including people, and updates this model in realtime to agree with changes in its environment. This 3D model is a qualitative description of the world, and includes physics simulation capabilities so that effects of actions can be simulated before being executed in the real world. This allows the unmanned system to predict and plan its interactions with people. This paper describes the structure of this architecture, and provides examples and videos showing its performance in dynamic environments.