TY - GEN
T1 - Spectral-Spatial Adaptive Transformer Model for Hyperspectral Image Classification
AU - Wang, Dong
AU - Liu, Sitian
AU - Zhu, Chunli
N1 - Publisher Copyright:
© 2023 SPIE. All rights reserved.
PY - 2023
Y1 - 2023
N2 - Hyperspectral imagery (HSI) classification, with the goal of assigning an appropriate land cover label to each hyperspectral pixel, is a challenging part of hyperspectral remote sensing. Recently, convolutional neural network-based HSI classification methods have shown superior performance due to their excellent locally contextual modeling ability. However, the ability of these methods to obtain deep semantic features is limited, and the computational cost increases markedly as the number of layers increases. In this work, we propose a novel spectral-spatial adaptive transformer (SSAT) model to adapt a pre-trained model for effective HSI classification. The main architecture of SSAT is based on vision transformer, which could aggregate features at different levels. Furthermore, we have designed an adaptive encoder block including spectral adaption, spatial adaption, and joint adaptation to extract HSI features in the spectral-spatial domains. Finally, the classification map is obtained from the fully connected layer. Extensive experiments have been conducted to validate the effectiveness of the proposed SSAT compared with seven typical HSI classification methods. Results demonstrate that the key classification evaluation index overall accuracy (OA) outperforms other comparative methods by at least 2.03%. Classification maps reveal the superior visualization effect, demonstrating that SSAT is an efficient tool for HSI classification.
AB - Hyperspectral imagery (HSI) classification, with the goal of assigning an appropriate land cover label to each hyperspectral pixel, is a challenging part of hyperspectral remote sensing. Recently, convolutional neural network-based HSI classification methods have shown superior performance due to their excellent locally contextual modeling ability. However, the ability of these methods to obtain deep semantic features is limited, and the computational cost increases markedly as the number of layers increases. In this work, we propose a novel spectral-spatial adaptive transformer (SSAT) model to adapt a pre-trained model for effective HSI classification. The main architecture of SSAT is based on vision transformer, which could aggregate features at different levels. Furthermore, we have designed an adaptive encoder block including spectral adaption, spatial adaption, and joint adaptation to extract HSI features in the spectral-spatial domains. Finally, the classification map is obtained from the fully connected layer. Extensive experiments have been conducted to validate the effectiveness of the proposed SSAT compared with seven typical HSI classification methods. Results demonstrate that the key classification evaluation index overall accuracy (OA) outperforms other comparative methods by at least 2.03%. Classification maps reveal the superior visualization effect, demonstrating that SSAT is an efficient tool for HSI classification.
KW - Adaptive transformer
KW - Hyperspectral images classification
KW - Spatial-spectral joint adaptation
UR - http://www.scopus.com/inward/record.url?scp=85180128903&partnerID=8YFLogxK
U2 - 10.1117/12.2687107
DO - 10.1117/12.2687107
M3 - Conference contribution
AN - SCOPUS:85180128903
T3 - Proceedings of SPIE - The International Society for Optical Engineering
BT - Optoelectronic Imaging and Multimedia Technology X
A2 - Dai, Qionghai
A2 - Shimura, Tsutomu
A2 - Zheng, Zhenrong
PB - SPIE
T2 - Optoelectronic Imaging and Multimedia Technology X 2023
Y2 - 15 October 2023 through 16 October 2023
ER -