Visible/Infrared Image Registration Based on Region-Adaptive Contextual Multifeatures

Visible (VIS) and infrared image registration is a challenging problem in computer vision due to the significant differences in appearance and physical properties between the two modalities. A single feature is not enough to remove nonlinear differences, and the matching method faces a trade-off between the high-resolution feature map and the transformer model. In this article, we propose a novel method called adaptive-neighborhood contextual multifeatures (ANCM-Net) for VIS/infrared image registration. Our method addresses the limitations of existing approaches by incorporating depth features and cross-modal similar contour features to form contextual feature representations. Additionally, we propose a region-spanning adaptive cross-attention module to handle low spatial resolution and redundancy in attention computation. This module enables attentional encoding of limited information in the attention location and cross-modal adaptive region through attention region adjustment. In the matching task, we compute an adaptive attention region for each pixel point in the cross-modal image and encode and match the depth features and edge features together. As a result, ANCM-Net not only preserves the long-range dependency of the image feature structure but also achieves fine-grained attention between highly correlated pixels. By extracting cross-modal consistent contextual features to compensate for modality-specific information, our approach improves the cross-modal matching performance. Extensive experiments on real-world captured thermal infrared (TIR) and VIS datasets demonstrate that ANCM-Net outperforms existing image matching methods.