Boosting Domain Generalization in Remote Sensing Image Segmentation via Style Mapping and General Prototypical Contrast

Semantic segmentation models often suffer from significant performance degradation when applied to domains with different data distributions, particularly in unseen domains. This issue highlights a major challenge in improving the model’s generalization ability. We address this critical issue by enhancing the generalization of high-level semantic features and utilizing prior knowledge (i.e., style-related statistics) from the source domain embedded in the low-level features. In the context of domain generalization, the model is expected to aggregate general semantic concepts within high-level semantic features. To this end, we are the first to introduce a large vision-language model guided prototypical contrastive learning mechanism into the task of domain generalization for semantic segmentation. The proposed learning mechanism effectively leverages the general world knowledge encoded in the advanced vision-language model, such as CLIP, to guide the learning of generalized vision representations closely related to semantic segmentation. Style-related statistics obtained from low-level features capture domain-specific knowledge. Current methods are prone to neglect this style-related knowledge to obtain domain-invariant representation, resulting in sub-optimal generalization performance to an unseen domain. In contrast, we propose a lightweight style mapper which transfers the learned style-related prior knowledge from the source domain to the target domain using statistically derived style prototypes. In this way, domain-specific style-related knowledge is effectively utilized, leading to significantly better generalization to an unseen target domain. We validate the effectiveness of our approach through experiments conducted on several publicly available remote sensing image semantic segmentation datasets. Experimental results demonstrate significant improvements of our proposed method in terms of the model’s generalization to unseen domains.