Simultaneous hyperspectral image super-resolution and geometric alignment with a hybrid camera system

Existing snapshot hyperspectral cameras usually suffer from limited spatial resolution in order to maintain reasonable temporal and spectral resolution. In contrast, the spatial resolution of commercial RGB cameras is quite high. Therefore, many methods have been developed for hyperspectral image (HSI) super-resolution by fusing a high resolution RGB image and a low resolution HSI. These methods have been extensively evaluated by using simulated image pair with exact geometric alignment. However, the effect of misalignment on these methods, which always arises in hybrid camera system, has rarely been investigated. In this paper, we present an effective approach for simultaneous HSI super-resolution and geometric alignment of the image pair with drastically contrasting spatial resolution. Besides, we also conduct a systematic evaluation of the misalignment effect on five state-of-the-art hybrid HSI super-resolution methods under five different geometric transformations and three benchmark datasets. Experimental results on both synthetic data and real images show that the proposed method outperforms the current state-of-the-art HSI super-resolution methods with a misaligned hybrid camera system in terms of both objective metric and subjective visual quality.