MM-LLM for Depression: Towards an Augmented Intelligent Diagnosis and Intervention by Fusing Multimodal Data and Large Language Models

Depression is a highly prevalent mental disorder that remains difficult to assess and manage using traditional questionnaire- or interview-based approaches due to subjectivity and limited real-time adaptability. Existing support approaches further lack personalized interaction and adaptive feedback, limiting timely and individualized mental health support. To address these challenges, we propose MM-LLM, an augmented intelligent screening, assessment-support, and intervention-oriented response-generation framework by fusing multimodal data and Large Language Models (LLM). First, a cross-modal guidance module integrates EEG with speech and text representations using pretrained models to enhance neural discriminability. Second, a cross-domain knowledge transfer strategy aligns semantic spaces across subjects and task paradigms, enabling personalized yet generalizable modeling of depression-related features. Third, an LLM-based response-support module leverages state tracking, knowledge-graph retrieval, and retrieval-augmented generation to generate individualized supportive responses within a turn-level feedback cycle. Experimental results provide a proof-of-concept for MM-LLM's ability to enhance recognition accuracy and support response generation, while the small pilot pre-post evaluation provides only a preliminary short-term symptom-change signal that requires validation in larger controlled studies.