Spatial multi-semantic features guided spectral-friendly transformer network for hyperspectral image classification

Hyperspectral image classification (HSIC) is a foundational topic in remote sensing. However, the high correlations between bands and the spectral correlations often result in redundant data. Moreover, traditional convolutional neural networks (CNNs) compress spatial dimensions through pooling layers or strides during spatial information extraction, resulting in the loss of spatial information. To overcome these challenges, we propose a spatial multi-semantic features guided spectral-friendly Transformer network (SFTN), which effectively extracts the spectral and spatial features of HSIs. Specifically, a multi-semantic spatial attention (MsSA) module applies unidirectional spatial compression along the height and width dimensions. Thus, this module maintains spatial structure in one direction while aggregating global spatial information, thereby minimizing information loss during compression. It then employs multi-scale depth-shared 1D convolutions to capture multi-semantic spatial information. Furthermore, the spectral-friendly Transformer replaces the traditional multi-head self-attention (MHSA) with spectral correlation self-attention (ECSa), which effectively captures spectral differences and thus reduces the redundancy of spectral information. Extensive experiments on several HSI datasets show that the proposed SFTN method outperforms other state-of-the-art methods in HSIC applications. The source code for this work will be released later.