Illumination Conditions Adaptation for Data-Driven Keypoint Detection under Extreme Lighting Variations

Abstract

In the domain of robotics vision, feature detection under challenging illumination conditions, especially in low-light or night settings, has emerged as a critical task. This paper presents a novel method that improves the repeatability and robustness of feature extraction in such challenging scenarios. The proposed approach introduces the concept of Illumination Conditions Adaptation (ICA); a technique that utilizes our Photo-Realistic Synthetic Illumination (PRSI) dataset. This dataset was created in a state-of-the-art game engine using ultra-high resolution textures and raytracing to resemble realistic environments in urban settings. It consists of image sequences of the same scenes captured with a script-based camera under different lighting conditions, which are then leveraged to enhance the training procedure of a feature detector based on Deep Neural Networks. Specifically, the ICA operates under a self-supervised scheme, harnessing ground truths extracted from day images and imposing them onto the corresponding night samples in order to minimize artifacts and errors in the detection process. Our experiments validate the efficacy of our approach, providing a notable improvement in feature detection reliability, even under extremely low lighting conditions, while also increasing the number of accurate keypoint matches. This development holds the potential to significantly enhance the performance of robotics vision applications in a wide range of real-world scenarios.