Investigation of User Performance in Virtual Reality-based Annotation-assisted Remote Robot Control

Abstract

This poster investigates the use of point cloud processing algorithms to provide annotations for robotic manipulation tasks completed remotely via Virtual Reality (VR). A VR-based system has been developed that receives and visualizes the processed data from real-time RGB-D camera feeds. A real-world robot model has also been developed to provide realistic reactions and control feedback. The targets and the robot model are reconstructed in a VR environment and presented to users in different modalities. The modalities and available information are varied between experimental settings, and the associated task performance is recorded and analyzed. The results accumulated from 192 experiments completed by 8 participants showed that point cloud data is sufficient for completing the task. Additional information, either image stream or preliminary processes presented as annotations, was found to not have a significant impact on the completion time. However, the combination of image stream and colored point cloud data visualization modalities was found to greatly enhance a user’s performance accuracy, with the number of target centers missed being reduced by 40%.