Improved Situational Awareness in ROS Using Panospheric Vision and Virtual Reality

Abstract

One of the main difficulties in teleoperated systems is providing an operator with sufficient Situational Awareness (SA). This paper introduces three open-source packages that improve the operator's SA using the Robot Operating System (ROS). The first package-rviz_textured_sphere-allows rendering panospheric camera outputs as spherical images in the ROS visualization software RViz. A system where the visualization of this spherical data using an open-source virtual reality (OSVR) headset in the ROS framework is achieved with the second package: rviz_plugin_osvr. Finally, the third package-pointcloud_painter-projects spherical data onto a 3D depth cloud scan of the scene generated from a rotating lidar. This package outputs a XYZRGB pointcloud that can be visualized either in RViz or using the virtual reality headset. Together, these technologies address the wider issue of limited SA in robotics and represent a substantial advancement in the environment visualization capabilities available to open-source robotics developers.