Sparse tensor neighbor embedding based pan-sharpening via N-way block pursuit

Most of the available pan-sharpening methods use vector or matrix based detail injection to enhance the resolution of MultiSpectral (MS) image, which may result in unavoidable spectral and spatial distortions. In this paper we explore the intrinsic tensor structure and local sparsity of MS images, to develop a novel Sparse Tensor Neighbor Embedding (STNE) based pan-sharpening method that reduces the distortions in the fused images. First, MS images are formulated as some spectral tensors, and each tensor and its nearest neighbor tensors are assumed to lie in a low-dimensional manifold. Then the tensor is sparsely coded under its neighbor tensors, and a joint sparse coding assumption is cast on bands to develop an N-way Block Pursuit algorithm for solving sparse tensor coefficients. Finally high resolution MS tensor can be obtained by weighting Panchromatic image with the sparse tensor coefficients. Tensors are higher order generalizations of vectors and matrices, and taking advantage of high-order structure of multi-dimensional data can help us understand them. The proposed method first combines a sparse tensor with neighbor embedding, to construct a new high-dimensional sparse tensor embedding for efficient pan-sharpening. Because tensor formulation can exploit the structural correlations in high-dimensional MS data, the proposed method can well preserve spectral correlation among different bands simultaneously and capture the underlying high-order statistical properties of MS image. Some experiments are performed on several real QuickBird and GeoEye datasets, and experimental results show that STNE is superior to its counterparts in reducing spectral and spatial distortions.