Virtual Reality to Simulate Visual Tasks for Robotic Systems

Abstract

Virtual reality (VR) can be used as a tool to analyze the interactions between the visual system
\nof a robotic agent and the environment, with the aim of designing the algorithms to solve the
\nvisual tasks necessary to properly behave into the 3D world. The novelty of our approach lies
\nin the use of the VR as a tool to simulate the behavior of vision systems. The visual system of
\na robot (e.g., an autonomous vehicle, an active vision system, or a driving assistance system)
\nand its interplay with the environment can be modeled through the geometrical relationships
\nbetween the virtual stereo cameras and the virtual 3D world. Differently from conventional
\napplications, where VR is used for the perceptual rendering of the visual information to a
\nhuman observer, in the proposed approach, a virtual world is rendered to simulate the actual
\nprojections on the cameras of a robotic system. In this way, machine vision algorithms can be
\nquantitatively validated by using the ground truth data provided by the knowledge of both
\nthe structure of the environment and the vision system.