TY - GEN
T1 - HiTF
T2 - 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, KDD 2026
AU - Zhang, Yujuan
AU - Li, Qing
AU - Li, Xiuxing
AU - Wang, Zhuo
AU - Li, Ziyu
AU - Luo, Xiangqi
AU - Wu, Xia
N1 - Publisher Copyright:
© 2026 Owner/Author.
PY - 2026/8/8
Y1 - 2026/8/8
N2 - Multimodal sentiment analysis with incomplete data faces significant challenges, particularly in the form of random frame-level missingness, where fragmented emotional cues and heterogeneous data quality impede effective understanding. Existing completion methods predominantly rely on cross-modal consistency, often neglecting residual modality specificity and failing to assess cross-modal reliability, which leads to redundancy. Human cognitive systems exhibit remarkable robustness to incomplete sensory inputs, driven by two functional mechanisms: hippocampal memory systems that reconstruct missing content via pattern completion, and thalamic perceptual regulation that dynamically modulates multisensory integration by filtering unreliable signals. Inspired by these brain functions, we propose the Hippocampal-Thalamic Function-Inspired Framework (HiTF). This framework integrates two complementary streams. The hippocampal-inspired intra-modal enhancement stream employs memory retrieval and sparse activation mechanisms to mine modality-specific semantics and reconstruct missing information. Concurrently, the thalamic-inspired inter-modal regulation stream evaluates the reliability of each modality to adaptively integrate high-quality cross-modal information while suppressing redundant interference. Comprehensive experiments on MOSI, MOSEI, and SIMS demonstrate that HiTF achieves superior performance with 1.5% - 2.5% average accuracy improvements over state-of-the-art methods across all missing rates. Notably, even under an extreme 90% missing condition on MOSI, our framework achieves substantial improvements in F1 score over strong baselines, validating the effectiveness of the proposed brain function-inspired framework.
AB - Multimodal sentiment analysis with incomplete data faces significant challenges, particularly in the form of random frame-level missingness, where fragmented emotional cues and heterogeneous data quality impede effective understanding. Existing completion methods predominantly rely on cross-modal consistency, often neglecting residual modality specificity and failing to assess cross-modal reliability, which leads to redundancy. Human cognitive systems exhibit remarkable robustness to incomplete sensory inputs, driven by two functional mechanisms: hippocampal memory systems that reconstruct missing content via pattern completion, and thalamic perceptual regulation that dynamically modulates multisensory integration by filtering unreliable signals. Inspired by these brain functions, we propose the Hippocampal-Thalamic Function-Inspired Framework (HiTF). This framework integrates two complementary streams. The hippocampal-inspired intra-modal enhancement stream employs memory retrieval and sparse activation mechanisms to mine modality-specific semantics and reconstruct missing information. Concurrently, the thalamic-inspired inter-modal regulation stream evaluates the reliability of each modality to adaptively integrate high-quality cross-modal information while suppressing redundant interference. Comprehensive experiments on MOSI, MOSEI, and SIMS demonstrate that HiTF achieves superior performance with 1.5% - 2.5% average accuracy improvements over state-of-the-art methods across all missing rates. Notably, even under an extreme 90% missing condition on MOSI, our framework achieves substantial improvements in F1 score over strong baselines, validating the effectiveness of the proposed brain function-inspired framework.
KW - brain-inspired framework
KW - hippocampal-thalamic function
KW - incomplete data
KW - multimodal sentiment analysis
UR - https://www.scopus.com/pages/publications/105051046431
U2 - 10.1145/3770855.3818011
DO - 10.1145/3770855.3818011
M3 - Conference contribution
AN - SCOPUS:105051046431
T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
SP - 6665
EP - 6676
BT - KDD 2026 - Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2
PB - Association for Computing Machinery
Y2 - 9 August 2026 through 13 August 2026
ER -