Affective computing methods for multimodal embodied AI human–computer interaction

Purpose – In the context of rapid innovation in computing technology, the Embodied Artificial Intelligence (EAI) design methods and key research issues continuously evolve. Affective computing is now becoming a core means to assits in EAI design research, enriching the emotional connotations of EAI design and promoting more natural and harmonious human–computer interaction. Design/methodology/approach – This paper adopted a systematic literature review approach and ultimately selected 36 high-quality papers related to affective computing methods in multimodal EAI human–computer interaction. The analysis is conducted on the temporal migration of research topics and the distribution of four aspects: research content, application areas, data collection form and algorithm. Findings – Three key steps were outlined in the study of affective computing methods in multimodal EAI human–computer interaction: Emotional recognition in human–computer interaction, evolutionary learning of user characteristic emotional classification, emotional analysis and behavioral feedback. Practical implications – The paper prospects the application scenarios of affective computing assistance in embodied intelligent human–computer interaction, deriving trends in enhancing emotional experience in medical service design, providing simulation training for dangerous engineering environments, supplying social design with safety perception and mitigating the socio-ethical challenges raised by the new technology. Social implications – Four challenges in the emotional data collection process were summarized: privacy protection issues, questions about the accuracy and credibility of generative design, embodiment interaction dilemmas in cross-cultural contexts and social-ethical issues surrounding affective computing assistance in EAI. Originality/value – Previous studies mainly focused on the algorithmic effects of EAI and the design of a single modality experience. This paper offers an in-depth analysis from a multimodal viewpoint, following the research framework of affective computing. It presents a thorough examination of the human-centered interaction experience facilitated by embodied intelligent design.