Trusted multi-factor fusion: A quantitative model for the confidence of naturalistic stimuli and multimodal neural responses

Naturalistic paradigms have emerged as a pivotal methodology in emotion-related research due to their ecological validity and alignment with real-world experiences. Despite their widespread adoption, current studies employing physiological signals under such paradigms face two critical challenges: the lack of confidence quantification for emotional materials and the effective integration of central and autonomic nervous system responses. To address these issues, we propose the Trusted Multi-Factor Fusion (TMFF) model, which objectively measures the confidence of naturalistic emotional stimuli and dynamically fuses multimodal neural factors, bridging these methodological gaps and enabling physiologically grounded, interpretable emotion decoding. First, we construct high-order tensors (for EEG/MEG) and feature matrices (for ECG) to capture spatiotemporal neural dynamics. A joint tensor-matrix decomposition method is then applied to extract shared and modality-specific neural factors, reducing redundancy and enhancing complementary information utilization. Further, subjective logic theory and Dempster-Shafer combination rules are employed to quantify and fuse the confidence of multimodal factors in emotion decoding, guided by task-specific labels. Experimental results demonstrate that TMFF effectively provides interpretable insights into stimulus efficacy and modality-specific contributions and significantly improves emotion recognition accuracy.