Real-time egocentric superimposition of operator's own body on telexistence avatar in virtual environment

Abstract

During teleoperation manipulation, the synchronization of the user behavior and a remote avatar is important to deliver the sensation of being in that remote place. Current telexistence technologies allow full upper body posture synchronization through multi-DOF humanoid robot structures and allow the operator to control the remote body as his own. However, it does not preserve a consistent feedback, such as the human like skin tones, operator's hand shape and the current outfit he is wearing during the operation. Thus in this paper we propose a new method that provides the operator's body shape, complexion, and light correction using real-time visuals taken from a see-through camera placed in the HMD and superimposed over robot vision. By using hand and arm trajectory from a kinematics solver, a virtual representation is used to generate masking images that isolate his local body appearance and superimpose it into the virtual environment. Local body appearance is captured via a see-through HMD. This paper describes the design and implementation of the above technique and obtained basic results.