跳到主要导航 跳到搜索 跳到主要内容

HiTF: A Hippocampal-Thalamic Function-Inspired Framework for Robust Multimodal Sentiment Analysis with Incomplete Data

  • Beijing Institute of Technology
  • University College London

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

Multimodal sentiment analysis with incomplete data faces significant challenges, particularly in the form of random frame-level missingness, where fragmented emotional cues and heterogeneous data quality impede effective understanding. Existing completion methods predominantly rely on cross-modal consistency, often neglecting residual modality specificity and failing to assess cross-modal reliability, which leads to redundancy. Human cognitive systems exhibit remarkable robustness to incomplete sensory inputs, driven by two functional mechanisms: hippocampal memory systems that reconstruct missing content via pattern completion, and thalamic perceptual regulation that dynamically modulates multisensory integration by filtering unreliable signals. Inspired by these brain functions, we propose the Hippocampal-Thalamic Function-Inspired Framework (HiTF). This framework integrates two complementary streams. The hippocampal-inspired intra-modal enhancement stream employs memory retrieval and sparse activation mechanisms to mine modality-specific semantics and reconstruct missing information. Concurrently, the thalamic-inspired inter-modal regulation stream evaluates the reliability of each modality to adaptively integrate high-quality cross-modal information while suppressing redundant interference. Comprehensive experiments on MOSI, MOSEI, and SIMS demonstrate that HiTF achieves superior performance with 1.5% - 2.5% average accuracy improvements over state-of-the-art methods across all missing rates. Notably, even under an extreme 90% missing condition on MOSI, our framework achieves substantial improvements in F1 score over strong baselines, validating the effectiveness of the proposed brain function-inspired framework.

源语言英语
主期刊名KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
出版商Association for Computing Machinery
页6665-6676
页数12
ISBN(电子版)9798400722592
DOI
出版状态已出版 - 8 8月 2026
活动32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026 - Jeju Island, 韩国
期限: 9 8月 2026 → 13 8月 2026

丛书

姓名Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
卷2-B
ISSN(印刷版)2154-817X

会议

会议32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026
国家/地区韩国
市Jeju Island
时期9/08/26 → 13/08/26

学术指纹

探究 'HiTF: A Hippocampal-Thalamic Function-Inspired Framework for Robust Multimodal Sentiment Analysis with Incomplete Data' 的科研主题。它们共同构成独一无二的学术指纹。

引用此