Crossmodal Transformer on Multi-Physical Signals for Personalised Daily Mental Health Prediction

The rapid advancement of wearable sensors and machine learning technologies has opened new avenues for mental health monitoring. Despite these advancements, conventional approaches often fail to provide an accurate and personalised understanding of an individual's multi-dimensional emotional state. This paper introduces a novel approach for enhanced daily mental health prediction, focusing on nine distinct emotional states. Our method employs a personalised crossmodal transformer architecture that effectively integrates ZCM (Zero Crossing Mode) and PIM (Proportional Integration Mode) physical signals obtained from piezoelectric accelerometers worn on the non-dominant wrist. Utilising this personalised crossmodal transformer model, our approach adaptively focuses on the most pertinent features across these diverse physical signals, thereby offering a more nuanced and individualised assessment of an individual's emotional state. Our experiments show a considerable improvement in performance, achieving a Concordance Correlation Coefficient (CCC) of 0.475 over a baseline of 0.281.