Mapping stereo matching algorithms to hardware

Abstract

Stereo vision uses two cameras side by side and extracts the displacement of the objects caused by the cameras' different viewpoints.The displacement, called disparity, is directly correlated to the distance of the objects.This way a three-dimensional depth map can be computed.The core element of a stereo vision system, the stereo matching algorithm, has a high computational complexity.Fortunately, area-based algorithms proved to be very suitable for solutions using hardware-based parallel processing, enabling the implementation of real-time embedded stereo vision systems with high frame rates.In robotic applications, embedded stereo vision systems are likely to operate in an environment that is subject to repeated changes.However, in contrast to software-based implementations, hardware-based solutions cannot easily change their behavior without deactivating parts of their resources.Therefore, a novel technique to adapt block size, disparity range and frame rate for hardware implementations of area-based stereo matching algorithms is proposed.This technique is suitable for FPGAs as well as ASICs, i.e., it comes along without dynamic reconfiguration, enabling its implementation on real-time systems with short deadlines.Furthermore, novel approaches on how the accuracy of real-time algorithms can be improved are presented in this work.These approaches are not limited to the extension of existing algorithms for enhanced noise robustness, but also includes the presentation of a novel highly accurate real-time stereo matching algorithm that achieves an accuracy that until now was achievable with non-real time algorithms only, while still being apt to hardware-based solutions.This novel stereo matching algorithm is targeted at a high accuracy in non-occluded regions, i.e. all image areas where a stereo correspondence can be found.The evaluation of this algorithm using the Middlebury Stereo Ranking, at a maximum deviation of half a pixel, shows, that by the time this work was performed, it is the only real-time algorithm that is capable of challenging state of the art non real-time algorithms.