CMD³: Cross-Modal Decoupled Deformable Distillation for EEG-fNIRS Fusion

Multimodal fusion of Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) has shown great promise in Brain-Computer Interface (BCI) tasks. However, due to differences in physical mechanisms, temporal dynamics, and semantic representations between the two modalities, the fusion process faces significant challenges such as heterogeneity and temporal misalignment. To address this, we propose a cross-modal decoupled deformable distillation (CMD³) method, which aims to achieve flexible, efficient, and interpretable EEG-fNIRS fusion learning. CMD³ first decouples the feature representations of each modality into modality-independent and modality-specific spaces to separately model commonality and complementary information. A deformable feature extraction network is then designed to process shared and specific features individually, enabling cross-modal temporal alignment via predicted dynamic offsets, thereby mitigating response delays between modalities. Furthermore, to facilitate inter-modal knowledge transfer, we construct a dual-space graph distillation module to explicitly migrate semantic information across modalities, with learnable edge weights used to adaptively regulate the distillation strength. CMD³ is systematically evaluated on public datasets covering emotion recognition and motor imagery tasks. Experimental results demonstrate that CMD³ consistently outperforms existing fusion approaches in classification performance. Offset visualization further reveals physiologically meaningful temporal attention patterns learned by the model, validating the effectiveness and explainability of the proposed method.