TY - JOUR
T1 - Automatic Assessment Method and Device for Depression Symptom Severity Based on Emotional Facial Expression and Pupil-Wave
AU - Li, Mi
AU - Lu, Zeying
AU - Cao, Qishuang
AU - Gao, Junlong
AU - Hu, Bin
N1 - Publisher Copyright:
© 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
PY - 2024
Y1 - 2024
N2 - Depression is a serious mental disorder, significantly burdens individuals, families, and society. For clinical psychiatrists, assessing the severity of depression is a crucial tool in selecting treatment approaches and evaluating their effectiveness. Although many studies in machine learning have focused on the automatic evaluation of self-rating scales [such as the Beck Depression Inventory-II (BDI-II) and the Patient Health Questionnaire-8 (PHQ-8)], research into the machine learning-based automatic evaluation of the medical clinical assessment scale [such as the Hamilton Depression Scale (HAMD)] has not yet been focused on. In this study, an end-to-end automatic evaluation device for HAMD and Patient Health Questionnaire-9 (PHQ-9) scores was developed. In addition, we constructed a dataset consisting of emotional facial expression videos (eFEVs) signals and emotional pupil-wave (ePW) signals from 65 patients with depression. The dataset has HAMD and PHQ-9 score labels, encompassing two emotional states: sadness and happiness. We built a 3-dimensional convolutional neural network + long short-term memory (3DCNN + LSTM) model framework and a multiscale 1-dimensional convolutional neural network (1DCNN) to learn and extract features from eFEVs and ePW automatically. The results showed that compared with the previous evaluation methods for depression levels, the evaluation precision of HAMD and PHQ-9 has been improved significantly. The results also showed that, in both HAMD and PHQ-9 evaluations, the evaluation precision of eFEVs was superior to ePW, and HAMD is better than PHQ-9. These studies indicated that both emotional facial expressions and ePW can better represent depressive mood in patients with depression, especially emotional facial expressions, and the predictive precision of the medical scale is significantly better than the self-rating scale. This automated assessment method and device can assist doctors in diagnosing depressive symptoms more effectively and serve as an evaluation tool for treatment efficacy.
AB - Depression is a serious mental disorder, significantly burdens individuals, families, and society. For clinical psychiatrists, assessing the severity of depression is a crucial tool in selecting treatment approaches and evaluating their effectiveness. Although many studies in machine learning have focused on the automatic evaluation of self-rating scales [such as the Beck Depression Inventory-II (BDI-II) and the Patient Health Questionnaire-8 (PHQ-8)], research into the machine learning-based automatic evaluation of the medical clinical assessment scale [such as the Hamilton Depression Scale (HAMD)] has not yet been focused on. In this study, an end-to-end automatic evaluation device for HAMD and Patient Health Questionnaire-9 (PHQ-9) scores was developed. In addition, we constructed a dataset consisting of emotional facial expression videos (eFEVs) signals and emotional pupil-wave (ePW) signals from 65 patients with depression. The dataset has HAMD and PHQ-9 score labels, encompassing two emotional states: sadness and happiness. We built a 3-dimensional convolutional neural network + long short-term memory (3DCNN + LSTM) model framework and a multiscale 1-dimensional convolutional neural network (1DCNN) to learn and extract features from eFEVs and ePW automatically. The results showed that compared with the previous evaluation methods for depression levels, the evaluation precision of HAMD and PHQ-9 has been improved significantly. The results also showed that, in both HAMD and PHQ-9 evaluations, the evaluation precision of eFEVs was superior to ePW, and HAMD is better than PHQ-9. These studies indicated that both emotional facial expressions and ePW can better represent depressive mood in patients with depression, especially emotional facial expressions, and the predictive precision of the medical scale is significantly better than the self-rating scale. This automated assessment method and device can assist doctors in diagnosing depressive symptoms more effectively and serve as an evaluation tool for treatment efficacy.
KW - Deep learning
KW - Hamilton Depression Scale (HAMD)
KW - Patient Health Questionnaire-9 (PHQ-9)
KW - patients with depression
KW - symptom severity
UR - http://www.scopus.com/inward/record.url?scp=85200794396&partnerID=8YFLogxK
U2 - 10.1109/TIM.2024.3415778
DO - 10.1109/TIM.2024.3415778
M3 - Article
AN - SCOPUS:85200794396
SN - 0018-9456
VL - 73
JO - IEEE Transactions on Instrumentation and Measurement
JF - IEEE Transactions on Instrumentation and Measurement
M1 - 2531215
ER -