TY - JOUR
T1 - Cross-modal Prompt Disentangled Graph Neural Networks for incomplete conversational emotion recognition
AU - Qiao, Shi
AU - Zhang, Xiaowei
AU - Zhao, Qinglin
AU - Wang, Bimei
AU - Dang, Jisheng
AU - Hu, Bin
AU - Peng, Hong
N1 - Publisher Copyright:
© 2026 Elsevier B.V.
PY - 2026/9/27
Y1 - 2026/9/27
N2 - Multimodal emotion recognition relies on the joint modeling of text, speech, and visual signals. In real-world scenarios, however, some modalities are often unavailable due to noise, occlusion, recognition errors, or device malfunction. This not only weakens the use of complementary cross-modal information, but also intensifies imbalance in modality learning. To address this issue, we propose a Cross-modal Prompt Disentangled Graph Neural Network (CPD-GNN) for incomplete multimodal conversational emotion recognition. Specifically, to reduce interference caused by modality heterogeneity, the proposed method first projects incomplete multimodal features into a unified semantic space and disentangles them into shared representations and modality specific representations. On this basis, the framework jointly performs cross-modal information compensation and dynamic modality balancing, providing a robust foundation for modeling both local contextual interactions and higher-order conversational dependencies under missing-modality conditions. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method is effective for incomplete multimodal learning and achieves competitive or superior performance compared with existing approaches.
AB - Multimodal emotion recognition relies on the joint modeling of text, speech, and visual signals. In real-world scenarios, however, some modalities are often unavailable due to noise, occlusion, recognition errors, or device malfunction. This not only weakens the use of complementary cross-modal information, but also intensifies imbalance in modality learning. To address this issue, we propose a Cross-modal Prompt Disentangled Graph Neural Network (CPD-GNN) for incomplete multimodal conversational emotion recognition. Specifically, to reduce interference caused by modality heterogeneity, the proposed method first projects incomplete multimodal features into a unified semantic space and disentangles them into shared representations and modality specific representations. On this basis, the framework jointly performs cross-modal information compensation and dynamic modality balancing, providing a robust foundation for modeling both local contextual interactions and higher-order conversational dependencies under missing-modality conditions. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method is effective for incomplete multimodal learning and achieves competitive or superior performance compared with existing approaches.
KW - Conversational emotion recognition
KW - Incomplete multimodal learning
KW - Multimodal fusion
UR - https://www.scopus.com/pages/publications/105043597125
U2 - 10.1016/j.knosys.2026.116547
DO - 10.1016/j.knosys.2026.116547
M3 - Article
AN - SCOPUS:105043597125
SN - 0950-7051
VL - 350
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 116547
ER -