ND-MRM: Neuronal Diversity Inspired Multisensory Recognition Model

Qixin Wang, Chaoqiong Fan, Tianyuan Jia, Yuyang Han, Xia Wu*

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

Abstract

Cross-sensory interaction is a key aspect of multisensory recognition. Without cross-sensory interaction, artificial neural networks show inferior performance in multisensory recognition. On the contrary, the human brain has an inherently remarkable ability in multisensory recognition, which stems from the diverse neurons that exhibit distinct responses to sensory inputs, especially the multisensory neurons with multisensory responses hence enabling cross-sensory interaction. Based on this neuronal diversity, we propose a Neuronal Diversity inspired Multisensory Recognition Model (ND-MRM), which, similar to the brain, comprises unisensory neurons and multisensory neurons. To reflect the different response characteristics of diverse neurons in the brain, special connection constraints are innovatively designed to regulate the feature transmission in the ND-MRM. Leveraging this novel concept of neuronal diversity, our model is biologically plausible, enabling more effective recognition of multisensory information. To validate the performance of the proposed ND-MRM, we employ a multisensory emotion recognition task as a case study. The results demonstrate that our model surpasses state-of-the-art brain-inspired baselines on two datasets, proving the potential of brain-inspired methods for advancing multisensory interaction and recognition.

Original languageEnglish
Pages (from-to)15589-15597
Number of pages9
JournalProceedings of the AAAI Conference on Artificial Intelligence
Volume38
Issue number14
DOIs
Publication statusPublished - 25 Mar 2024
Externally publishedYes
Event38th AAAI Conference on Artificial Intelligence, AAAI 2024 - Vancouver, Canada
Duration: 20 Feb 202427 Feb 2024

Fingerprint

Dive into the research topics of 'ND-MRM: Neuronal Diversity Inspired Multisensory Recognition Model'. Together they form a unique fingerprint.

Cite this