Adversarial Example Generation With Pseudo-Siamese Adversarial Generative Networks for Multimodal Remote Sensing Images

In the field of remote sensing, the increasing diversity of remote sensing image modalities has made the integration of multimodal remote sensing image information a prevailing trend to increase classification accuracy. Concurrently, the study of adversarial samples for multimodal remote sensing images has emerged as a crucial area for enhancing network robustness. However, existing adversarial attack strategies designed for single-modal data often fail to extend effectively to multimodal adversarial attack tasks, mainly due to the following challenges: Multimodal correlation: Since multimodal data provide complementary auxiliary information, attacking a single modality alone cannot disrupt the correlated features across modalities; directional differences in multimodal adversarial samples: The adversarial perturbation directions exhibit substantial discrepancies and conflicts, which considerably diminish the overall attack efficacy. To address the first challenge, we propose a pseudo-Siamese generative adversarial network that employs modality-specific generators to simultaneously produce perturbations for each modality from the latent feature space, enabling simultaneous attacks on multiple modalities. To address the second challenge, we introduce a collaborative adversarial loss that enforces consistency in the perturbation directions across modalities, thereby mitigating the conflicts between multimodal perturbations and improving attack effectiveness on multimodal classification networks. Extensive experiments demonstrate the vulnerability of multimodal fusion models to adversarial attacks, even when only a single modality is attacked. Specifically, we show that our proposed pseudo-Siamese adversarial attack method considerably reduces the overall accuracy of the U-Net and Deeplabv3 models from 81.92% and 82.20% to 0.22% and 4.16%, respectively, thereby validating the efficacy of our approach.