Issues in Control of a Robotic Spatial Augmented Reality System

Abstract

A robotic spatial augmented reality (RSAR) system combines a robotics technology with a spatial augmented reality system (SAR) where cameras are used to recognize real objects and projectors augment information and user interface directly on the surface of the recognized objects, rather than relying on handheld display devices. Moreover, a robotic module is actively used to discover and utilize the context of users and environments. The control of a RSAR system involves several issues from different technical fields such as classical inverse kinematics of motors where projector-camera pairs are mounted, inverse projection problems to find appropriate internal/external parameters of projectors and cameras, and image warping in graphics pipeline to compensate the kinematic constraints. In this paper, we investigate various control issues related to a RSAR system and propose basic approaches to handle them, specially focused on the prototype RSAR system developed in ETRI.