ADED: Method and Device for Automatically Detecting Early Depression Using Multimodal Physiological Signals Evoked and Perceived via Various Emotional Scenes in Virtual Reality

Mi Li*, Yanbo Chen, Zeying Lu, Fan Ding, Bin Hu*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

Severe depression often exhibits suicidal tendencies, making early identification and intervention crucial to prevent its further progression. This study focuses on developing a high-performance method and device for early detection of depression. We propose a hybrid framework for early depression detection that integrates multiple deep learning techniques and ensemble learning. This framework features a dual bidirectional temporal convolutional network (BiTCN) to encode both local and global causal relationships, a bidirectional long short-term memory (BiLSTM) network to capture long-term dependencies and contextual relationships, an emotional cross-attention (eCA) module to encode the significance of different emotions, a multimodal feature cross-attention (MFCA) mechanism to prioritize various feature modalities, and an ensemble learning method to decode and infer depression detection. The input signals include pupil waves and pulse rate variability (PRV) signals, measured during both calm (non-emotional) and emotional states such as sadness, happiness, fear, and tension. To enhance the generalization capability of the model, data augmentation techniques are applied to the training dataset. Test results show that detection performance based on emotional cues (ECs) is superior to that based on non-ECs (calm). Notably, the fusion of pupil waves and PRV signals with ECs has achieved state-of-the-art (SOTA) performance in depression detection. These findings highlight the crucial role of emotional signals in improving the performance of depression detection. The end-to-end high-performance automatic detection of early depression (ADED) device developed in this study can serve as an early detection tool, thereby promoting the potential application of artificial intelligence technology in mental health screening and clinical practice.

Original languageEnglish
Article number2524016
JournalIEEE Transactions on Instrumentation and Measurement
Volume74
DOIs
Publication statusPublished - 2025
Externally publishedYes

Keywords

  • Deep learning
  • depression
  • emotion
  • pulse rate variability (PRV)
  • pupil wave

Fingerprint

Dive into the research topics of 'ADED: Method and Device for Automatically Detecting Early Depression Using Multimodal Physiological Signals Evoked and Perceived via Various Emotional Scenes in Virtual Reality'. Together they form a unique fingerprint.

Cite this