LNFormer: Lightweight Design for Nighttime Semantic Segmentation With Transformer

Abstract

General image semantic segmentation methods mainly focus on daytime images with sufficient light, nighttime images have low contrast and blurred details compared to daytime images, and this difference makes it difficult for conventional semantic segmentation algorithms to effectively distinguish between the target and the background. Moreover, semantic segmentation needs to function in real time as part of visual perception. As a result, achieving high-quality segmentation of nighttime scenes while ensuring speed is a particularly challenging task. In this article, we present a lightweight design for nighttime semantic segmentation with transformer (LNFormer) to implement real-time semantic segmentation. Targeting the problem of blurred details and low contrast in nighttime images, LNFormer proposes to utilize the lightweight transformer module and token pyramid to construct global information and enhance the feature representation capability of the model in nighttime situations. Furthermore, LNFormer presents the mutual fusion module based on spatial attention under the coding and decoding system to further enhance the feature display of the target spatial location. Extensive experiments have shown that LNFormer outperforms other real-time methods in nighttime and daytime datasets, striking a new balance between speed and accuracy. Our proposed approach offers an innovative direction for nighttime semantic segmentation and has potential applications in technologies such as autonomous driving, surveillance, and robotics.