Cloud based VR system with immersive interfaces to collect multimodal data in human-robot interaction

Abstract

This paper presents a cloud based VR system with immersive interfaces to collect multimodal data in human-robot interaction and its applications. The proposed system enables a subject to log in to the VR space as an avatar and to naturally interact with a virtual robot by immersive interfaces. A head mounted display and a motion capture device provide immersive visualization and natural motion control in the VR system, respectively. The proposed VR system can simultaneously perform natural human-robot interaction in a VR space and collect visual, physical, and voice data during human-robot interaction by the immersive interfaces. Two application experiments to learn object's attributes and to learn communication protocol demonstrate the availability of the proposed system.