TY - JOUR
T1 - ADED
T2 - Method and Device for Automatically Detecting Early Depression Using Multimodal Physiological Signals Evoked and Perceived via Various Emotional Scenes in Virtual Reality
AU - Li, Mi
AU - Chen, Yanbo
AU - Lu, Zeying
AU - Ding, Fan
AU - Hu, Bin
N1 - Publisher Copyright:
© 1963-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Severe depression often exhibits suicidal tendencies, making early identification and intervention crucial to prevent its further progression. This study focuses on developing a high-performance method and device for early detection of depression. We propose a hybrid framework for early depression detection that integrates multiple deep learning techniques and ensemble learning. This framework features a dual bidirectional temporal convolutional network (BiTCN) to encode both local and global causal relationships, a bidirectional long short-term memory (BiLSTM) network to capture long-term dependencies and contextual relationships, an emotional cross-attention (eCA) module to encode the significance of different emotions, a multimodal feature cross-attention (MFCA) mechanism to prioritize various feature modalities, and an ensemble learning method to decode and infer depression detection. The input signals include pupil waves and pulse rate variability (PRV) signals, measured during both calm (non-emotional) and emotional states such as sadness, happiness, fear, and tension. To enhance the generalization capability of the model, data augmentation techniques are applied to the training dataset. Test results show that detection performance based on emotional cues (ECs) is superior to that based on non-ECs (calm). Notably, the fusion of pupil waves and PRV signals with ECs has achieved state-of-the-art (SOTA) performance in depression detection. These findings highlight the crucial role of emotional signals in improving the performance of depression detection. The end-to-end high-performance automatic detection of early depression (ADED) device developed in this study can serve as an early detection tool, thereby promoting the potential application of artificial intelligence technology in mental health screening and clinical practice.
AB - Severe depression often exhibits suicidal tendencies, making early identification and intervention crucial to prevent its further progression. This study focuses on developing a high-performance method and device for early detection of depression. We propose a hybrid framework for early depression detection that integrates multiple deep learning techniques and ensemble learning. This framework features a dual bidirectional temporal convolutional network (BiTCN) to encode both local and global causal relationships, a bidirectional long short-term memory (BiLSTM) network to capture long-term dependencies and contextual relationships, an emotional cross-attention (eCA) module to encode the significance of different emotions, a multimodal feature cross-attention (MFCA) mechanism to prioritize various feature modalities, and an ensemble learning method to decode and infer depression detection. The input signals include pupil waves and pulse rate variability (PRV) signals, measured during both calm (non-emotional) and emotional states such as sadness, happiness, fear, and tension. To enhance the generalization capability of the model, data augmentation techniques are applied to the training dataset. Test results show that detection performance based on emotional cues (ECs) is superior to that based on non-ECs (calm). Notably, the fusion of pupil waves and PRV signals with ECs has achieved state-of-the-art (SOTA) performance in depression detection. These findings highlight the crucial role of emotional signals in improving the performance of depression detection. The end-to-end high-performance automatic detection of early depression (ADED) device developed in this study can serve as an early detection tool, thereby promoting the potential application of artificial intelligence technology in mental health screening and clinical practice.
KW - Deep learning
KW - depression
KW - emotion
KW - pulse rate variability (PRV)
KW - pupil wave
UR - http://www.scopus.com/inward/record.url?scp=105003088430&partnerID=8YFLogxK
U2 - 10.1109/TIM.2025.3551439
DO - 10.1109/TIM.2025.3551439
M3 - Article
AN - SCOPUS:105003088430
SN - 0018-9456
VL - 74
JO - IEEE Transactions on Instrumentation and Measurement
JF - IEEE Transactions on Instrumentation and Measurement
M1 - 2524016
ER -