SG-AFN: Structural Feature-guided Multispectral and Hyperspectral Image Alignment and Fusion

The fusion of high-resolution multispectral images (HR MSI) and low-resolution hyperspectral images (LR HSI) provides a cost-effective solution for acquiring high-quality hyperspectral images. However, most existing methods heavily rely on precise registration between LR HSI and HR MSI. Although some approaches attempt to jointly perform registration and fusion of LR HSI and HR MSI, their performance in practical scenarios remains suboptimal due to limitations such as occlusion sensitivity, cross-scale mapping ambiguity, and spectral response dependency. This paper proposes a structural feature-guided unregistered multispectral-hyperspectral fusion method, which overcomes the limitations of traditional pixel-level registration paradigms by: 1) circumventing explicit registration requirements through structural consistency constraints; 2) resolving cross-resolution mapping issues via the scale-invariant properties of structural features; and 3) achieving end-to-end cross-domain fusion in the feature space without requiring prior knowledge of spectral response functions (SRF). Specifically, our framework first extracts structural features from multispectral and hyperspectral images using simple gradient computation. We then design a dual-branch feature encoding network to capture multi-level structural features and texture features separately. Subsequently, a multi-scale fusion attention module is constructed to guide adaptive registration and fusion of texture features using structural similarity metrics. Finally, a decoder reconstructs the HR HSI from the fused features. Extensive experiments on both simulated and real-world datasets demonstrate that our method achieves significant improvements compared to state-of-the-art approaches.