TY - JOUR
T1 - Few-shot Remote Sensing Imagery Recognition with Compositionality Inductive Bias in Hierarchical Representation Space
AU - Zhou, Shichao
AU - Wang, Zhuowei
AU - Zhang, Zekai
AU - Wang, Wenzheng
AU - Zhao, Yingrui
AU - Zhang, Yunpu
N1 - Publisher Copyright:
© 2008-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Remote sensing scenes from aerial perspective can be constructed by distinct visual parts in a combinatorial number of different ways. Such combinatorial explosion poses great challenges to understanding remote sensing imagery (RSI) with few prior instances (i.e., few-shot RSI recognition). Despite empirical success of existing methods such as data augmentation and knowledge transfer, no large-scale dataset can cover all possible combinations of visual parts. In this case, the prior knowledge learned from these data-driven methods may exhibit dataset bias, resulting in inadequate generalization to the current recognition task. Different from the naive data-driven strategies mentioned above, we alternatively devote to delicate feature modeling by constraining the mapping behavior of deep neural networks. Specifically, we embed inductive bias of compositionality into hierarchical latent representation space, which operates on two aspects: 1) disentangled and reusable representation. We establish a clustering-oriented factorized representation with a mixture model to represent multipart distributions of tokens. Each cluster centroid represents a re-occurring part. New patches are allocated to the nearest cluster centroid, and then we obtain the posterior representation; 2) compositional and discriminative representation. We introduce a hierarchical context prediction mechanism for compositional representation learning, utilizing a predictive NCE loss function to encourage global remote sensing scenes to accurately predict similar local parts, and thus automatically inferring compositional representations of high-level but discriminative latent concepts. Extensive experiments, including comparative experiments with SOTA, sensitivity evaluations, and ablation studies, demonstrate comparable or even superior performance of our method in few-shot RSI recognition.
AB - Remote sensing scenes from aerial perspective can be constructed by distinct visual parts in a combinatorial number of different ways. Such combinatorial explosion poses great challenges to understanding remote sensing imagery (RSI) with few prior instances (i.e., few-shot RSI recognition). Despite empirical success of existing methods such as data augmentation and knowledge transfer, no large-scale dataset can cover all possible combinations of visual parts. In this case, the prior knowledge learned from these data-driven methods may exhibit dataset bias, resulting in inadequate generalization to the current recognition task. Different from the naive data-driven strategies mentioned above, we alternatively devote to delicate feature modeling by constraining the mapping behavior of deep neural networks. Specifically, we embed inductive bias of compositionality into hierarchical latent representation space, which operates on two aspects: 1) disentangled and reusable representation. We establish a clustering-oriented factorized representation with a mixture model to represent multipart distributions of tokens. Each cluster centroid represents a re-occurring part. New patches are allocated to the nearest cluster centroid, and then we obtain the posterior representation; 2) compositional and discriminative representation. We introduce a hierarchical context prediction mechanism for compositional representation learning, utilizing a predictive NCE loss function to encourage global remote sensing scenes to accurately predict similar local parts, and thus automatically inferring compositional representations of high-level but discriminative latent concepts. Extensive experiments, including comparative experiments with SOTA, sensitivity evaluations, and ablation studies, demonstrate comparable or even superior performance of our method in few-shot RSI recognition.
KW - Clustering methods
KW - feature extraction
KW - knowledge representation
KW - prediction methods
KW - remote sensing
UR - http://www.scopus.com/inward/record.url?scp=85214423496&partnerID=8YFLogxK
U2 - 10.1109/JSTARS.2024.3524573
DO - 10.1109/JSTARS.2024.3524573
M3 - Article
AN - SCOPUS:85214423496
SN - 1939-1404
VL - 18
SP - 3544
EP - 3555
JO - IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
JF - IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
ER -