Feature-level loss for multispectral pan-sharpening with machine learning

Multispectral pan-sharpening plays an important role in providing earth observation with both high-spatial and high-spectral resolutions, and recently pan-sharpening with machine learning has been attracting broad interest. However, these algorithms minimizing the pixel-wise mean squared error, generally suffer from over-smoothed results that lack of high-frequency details in both spatial and spectral dimensions. In this paper, we propose to tackle this problem by shifting the learning loss from pixel-wise error to a higher-level feature loss. The new loss function, formulated by spatial structure similarity and spectral angle mapping, pushes the model to generate results that have similar feature representations with ground truth, rather than match with pixel-wise accuracy. Consequently, more realistic fusion results can be produced. Visual and quantitative analysis both demonstrate that our approach achieves better performance in comparison with state-of-the-art algorithms. Furthermore, experiments on high-level remote sensing task further confirm the superiority of the proposed method in real applications.