Human Emotion Decoding for Mandarin with Anti-Background Noise Mask and Deep Learning

Emotion recognition is essential for enhancing human-computer interaction, mental health diagnostics, and AI applications. It fosters empathetic communication and leads to improved mental health outcomes through personalized user experiences that boost engagement and satisfaction. Yet, current emotion recognition methodologies rely on sensors that are vulnerable to environmental factors, background noise, privacy concerns, and comfort issues, hindering effective emotion recognition and its engineering applications across various fields. To address this, we present a sound-to-electricity triboelectric sensor (S2E-TENG) integrated into an N95 mask, called the Anti-Background Noise Mask (anti-BGN-Mask). By optimizing design parameters of S2E-TENG, the anti-BGN-Mask effectively captures human voice frequencies while minimizing interference from high-frequency sounds, ensuring accurate detection of desired audio even in noisy settings. Additionally, the unique voltage signals generated by the anti-BGN-Mask enable the collection of speech from people while maintaining their privacy. Using this anti-BGN-Mask, we constructed a triboelectric emotional database in Mandarin (TENG-EMODB) through real people speaking and developed a deep learning model that can differentiate between six major emotions, achieving an average accuracy of 84.86% and an f1 value of 0.8511 through 7-fold cross-validation. This work enhances understanding of human emotions and supports advancements in responsive human-computer interactions, mental health diagnostics, and various practical applications in artificial intelligence.