Encouraging the Mutual Interact between Dataset-Level and Image-Level Context for Semantic Segmentation of Remote Sensing Image

Recently, semantic segmentation of remote sensing images has witnessed rapid advancement with the adoption of deep neural networks. Contextual cues, referring to the long-range correlation between pixels, are crucial for achieving accurate segmentation results, particularly for objects with less discriminative characteristics in these images. Currently, most studies are centered on incorporating contextual cues by aggregating context information at the dataset level or image level. However, current research often treats contextual cue modeling at the dataset-level and image level as independent procedures, neglecting the intrinsic correlation between these two feature levels. Consequently, the obtained contextual cues are suboptimal. This issue is particularly critical in the semantic segmentation of remote sensing images. To address this, we propose to encourage mutual interaction between dataset-level and image-level contextual cues. Firstly, we propose an interactive dataset-image context aggregation scheme to obtain complementary and consistent multilevel contextual cues. Additionally, we introduce a parallel feature interaction network (PFI-Net) that progressively extracts and fuses features across multiple layers, enabling effective integration of multilevel contexts. Furthermore, we introduce an enhanced shifted window-based cross-attention mechanism to augment model efficiency. Extensive experimental results on the widely used Vaihingen dataset, GaoFen-2 dataset, and instance segmentation in aerial images dataset (iSAID) effectively demonstrate the superiority of our proposed method over the other state-of-the-art methods.