Noise-aware deep learning algorithm for one-stage multispectral pedestrian detection

Abstract

Multispectral pedestrian detection has gained momentum in the research literature with its wide range of applications in car safety, video surveillance, and robotics. We introduce a noise-aware multispectral detector, called NAMPD, that is based on the one-stage YOLOv4 to produce fast and accurate detection. NAMPD merges the color and thermal streams at the decision level by a weighted sum, where the weight is computed using both the noise level and average luminance of a given image pair. We show that the noise level provides a distinction between nighttime and daytime and can be utilized to improve the detection performance. Experiments are carried out on the sanitized KAIST dataset. They demonstrate that NAMPD is able to achieve a log-average miss rate of 4.25%, which is better than that of the conventional IAF R-CNN, cross-modality interactive attention network, MSDS-RCNN, and MCFF. This mechanism can operate at a speed of 15 fps on the general GPU 1080 Ti, and, thus, is a viable option to be deployed in practical pedestrian detection systems.