Pervasive 'Calm' Perception for Autonomous Robotic Agents

Abstract

A major bottleneck in the realization of autonomous robotic agents performing complex manipulation tasks are the re-quirements that these tasks impose onto perception mech-anisms. There is a strong need to scale robot perception capabilities along two dimensions: First, the variations of appearances and perceptual properties that real-world ob-jects exhibit. Second, the variety of perceptual tasks, like categorizing and localizing, decomposing objects into their functional parts, perceiving the affordances they provide. This paper, addresses this need by organizing percep-tion into a two-stage process. First, a pervasive and ‘calm’ perceptual component runs continually and interprets the incoming image stream to form a general purpose hybrid (symbolic/sub-symbolic) belief state. This is used by the second component, the task-directed perception subsystem, to perform the respective perception tasks in a more in-formed way. We describe and discuss the first component and explain how it can manage realistic belief states, form a memory of past perceptual experiences, and compute valu-able perceptual attributes without delaying plan execution. It does so by exploiting that perception is not a one-shot task but rather a secondary task that is pervasively and calmly performed throughout the lifetime of the robot. We show system operating on a leading-edge manipulation platform.