CWPFormer: Toward High-Performance Visual Place Recognition for Robot With Cross-Weight Attention Learning

Abstract

As an important component of robot localization and navigation, visual place recognition (VPR) has made significant improvements in the past few decades. Most existing state-of-the-art VPR methods face the following challenges: 1) Aiming only for performance with ideal conditions while overlooking real-world conditions; 2) The existing VPR paradigm, struggling to reconcile the transfer gap between upstream pre-training and downstream fine-tuning; 3) Deeper networks produce high-dimensional parameters in the model training that results in lower efficient of models. To address those problems, we propose a high-performance visual place recognition framework for robot navigation tasks. Concretely, our framework is composed of three major modules: 1) based on vision transformer (ViT), we design a siamese Cross-weight Pyramid Transformer (CWPFormer) backbone for image feature extraction. First, we integrate feature reconstruction and content cognition by inserting a feature pyramid phase into pre-training. Second, we establish weight correlation and sharing between feature downsampling and upsampling that offers multi-stage supervision to fine-tuning. 2) We found that the attention map has high similarity between heads, and the high-dimensional data processing based on the ViT leads to computational redundancy. To cope with this problem, we present a cascaded hash attention (CHA) module to feed the hash attention head with different complete feature splits, which not only saves the computational cost but also improves the attention diversity. 3) Besides, we adopt a Bayesian learning scheme with a dynamically constructed similarity matrix to learn one-dimensional compact hash codes to improve recognition accuracy. Exhaustive experiments demonstrate the superiority of our proposed VPR approach on datasets and real-world environments. Our code is available: <uri>https://github.com/CV4RA/CWPFormer</uri>.