An fNIRS-Based Interpretable Framework for Recognizing and Predicting Social-Emotional State in Children with ASD under Multimodal Social Robot Intervention

Although Social Robots (SR) show great potential in supporting Social-Emotional Learning (SEL) for children with Autism Spectrum Disorder (ASD), the underlying neural mechanisms that drive social-emotional improvement remain largely unexplored. Furthermore, in order to achieve precise and efficient intelligent interventions, it is essential to develop technologies that can predict the social-emotional state of children with ASD in real time. Using experimental data from 43 children with ASD, this study employs functional Near-Infrared Spectroscopy (fNIRS) to investigate the neural mechanisms and behavioral prediction potential of SR intervention in SEL. First, we compare different intervention methods for SEL in children with ASD to identify the neurocognitive advantages of SR intervention. Next, we use Graph Theory to extract the topological properties of brain networks and propose a Stacking-based ensemble learning method. This method predicts the social-emotional state of children with ASD and provides adaptive, real-time interaction support for future SR intervention. In addition, the Shapley Additive Explanations (SHAP) method builds an interpretability framework for the Stacking model. This framework identifies biomarkers that influence the social-emotional state of children with ASD. This process improves both model accuracy and transparency. Results indicate that the SR intervention elicits significantly higher activation in key brain regions than the control group. Functional Connectivity (FC) analysis showed stronger synchronization between specific brain regions. Conversely, Effective Connectivity (EC) shows weakened directional coupling in specific pathways. Additionally, under the current small-sample setting, the Stacking-SHAP framework achieved better performance than the base learners and typical deep learning models in this study. This interpretable prediction framework provides support for closed-loop brain-computer interface systems. Future SR can further optimize intervention strategies through the neural feedback of children with ASD.