Unsupervised Blind Hyperspectral Super-Resolution for Unregistered Images

Hyperspectral images super-resolution (HSI-SR) aims to fuse low-resolution HSI (LR-HSIs) and high-resolution multispectral images (HR-MSIs) for high-resolution HSIs (HR-HSIs). Most existing methods require registered image pairs and prior knowledge of spectral response functions (SRFs), which requires effort to realize in practical applications. To overcome this limitation, this paper proposes an unsupervised blind HSI-SR method (UBHSI-SR) for unregistered HSIs and MSIs. UBHSI-SR consists of two unmixing branches, each having its own encoder while sharing the decoder. First, the HSI unmixing branch learns to predict abundance maps and learns precise endmember spectra. Then, the learnable SRF transfers LR-HSIs to the registered LR-MSIs. The abundance similarity constraint between LR-HSIs and LR-MSIs guides the learning of the MSI encoder. With the abundance maps of HR-MSI, the shared decoder predicts the HR-HSIs as final results. Experiments on three remote sensing datasets validate the superior performance of UBHSI-SR to existing fusion methods.