Skip to main navigation Skip to search Skip to main content

HiTF: A Hippocampal-Thalamic Function-Inspired Framework for Robust Multimodal Sentiment Analysis with Incomplete Data

  • Beijing Institute of Technology
  • University College London

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Multimodal sentiment analysis with incomplete data faces significant challenges, particularly in the form of random frame-level missingness, where fragmented emotional cues and heterogeneous data quality impede effective understanding. Existing completion methods predominantly rely on cross-modal consistency, often neglecting residual modality specificity and failing to assess cross-modal reliability, which leads to redundancy. Human cognitive systems exhibit remarkable robustness to incomplete sensory inputs, driven by two functional mechanisms: hippocampal memory systems that reconstruct missing content via pattern completion, and thalamic perceptual regulation that dynamically modulates multisensory integration by filtering unreliable signals. Inspired by these brain functions, we propose the Hippocampal-Thalamic Function-Inspired Framework (HiTF). This framework integrates two complementary streams. The hippocampal-inspired intra-modal enhancement stream employs memory retrieval and sparse activation mechanisms to mine modality-specific semantics and reconstruct missing information. Concurrently, the thalamic-inspired inter-modal regulation stream evaluates the reliability of each modality to adaptively integrate high-quality cross-modal information while suppressing redundant interference. Comprehensive experiments on MOSI, MOSEI, and SIMS demonstrate that HiTF achieves superior performance with 1.5% - 2.5% average accuracy improvements over state-of-the-art methods across all missing rates. Notably, even under an extreme 90% missing condition on MOSI, our framework achieves substantial improvements in F1 score over strong baselines, validating the effectiveness of the proposed brain function-inspired framework.

Original languageEnglish
Title of host publicationKDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
PublisherAssociation for Computing Machinery
Pages6665-6676
Number of pages12
ISBN (Electronic)9798400722592
DOIs
Publication statusPublished - 8 Aug 2026
Event32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026 - Jeju Island, Korea, Republic of
Duration: 9 Aug 2026 → 13 Aug 2026

Publication series

NameProceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Volume2-B
ISSN (Print)2154-817X

Conference

Conference32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026
Country/TerritoryKorea, Republic of
CityJeju Island
Period9/08/26 → 13/08/26

Keywords

  • brain-inspired framework
  • hippocampal-thalamic function
  • incomplete data
  • multimodal sentiment analysis

Fingerprint

Dive into the research topics of 'HiTF: A Hippocampal-Thalamic Function-Inspired Framework for Robust Multimodal Sentiment Analysis with Incomplete Data'. Together they form a unique fingerprint.

Cite this