Inertial sensor data integration in computer vision systems

Abstract

Advanced sensor systems,exploring hig integH41M of multiple sensorial modalities, have beensig44M tlyincreasing the capabilities of autonomous robots andenlargHM the application potential of vision systems. In this work I explore the cooperation betweenimag and inertial sensors, motivated by what happens with the vestibular system and vision in humans and animals. Visual and inertialsensing are two sensory modalities that can be explored to gM e robust solutions onimag seg tation and recovery of three-dimensional structure. In this work I overview the currently available low-cost inertial sensors.Using some of these sensors, I have built an inertial system prototype and coupled it to the vision system used in this work. The vision system has a set of stereo cameras with vergB14 Using the information about the vision system's attitude in space,ga en by the inertial sensors, I obtained some interesting results. I use the inertial information about the vertical to infer one of the intrinsic parameters of the visual sensor - the focal distance. The process involves having at least oneimag vanishing point, andtracing an artificial horizon. Based on the integHz1M of inertial and visual information, I was able to detect threedimensional world features such as theg round plane and vertical features.Relying on the known vertical reference, and a few system parameters, I was able to determine theg round planeg eometric parameters and the stereo pair mapping ofimag points that belong to theg round plane. This enabled thesegB tation and three-dimensional reconstruction of gMVz4 plane patches. It was also used to identify the three-dimensional vertical structures in a scene. Since the vertical reference does notg ive aheading imag vanishing points can be used as an external heading ...