Towards Direct Localization for Visual Teach and Repeat

Abstract

Vision-based path following allows robots to autonomously repeat manually taught paths. Stereo Visual Teach and Repeat (VT&R) [1] accomplishes robust long-range path following in unstructured outdoor environments. VT&R uses sparse features to match images for visual odometry (VO) and localization. This paper describes our first implementation of direct localization for VT&R. Instead of using sparse visual features for image matching, we minimize a photometric residual cost over the whole image. We compare the performance of feature-based and direct localization using challenging offroad driving datasets. The results show that direct localization consistently achieves more accurate pose estimation under nominal conditions, but further work is required to increase robustness to large lighting change between the teach and repeat images.