EmoSense: Emotion Recognition for Mobile Computing via Multimodal Headband

Integrating emotion recognition into wearable platforms is critical for mobile computing, yet existing approaches based on computer vision, audio, or wristband sensing suffer from privacy concerns, environmental sensitivity, or limited discriminative power. This paper presents EmoSense, a lightweight headband-based multimodal emotion recognition system tailored for mobile scenarios. We develop a compact headband integrating EEG, PPG, and sensors for synchronized acquisition of multimodal signals at 500 Hz. To enable efficient inference on resource-constrained platforms, we propose TED-Net (Triple-modal Emotion Detection Network), a lightweight architecture that employs modality-specific encoders to learn compact cross-modal representations. An end-to-end cloud-assisted pipeline supports continuous sensing and low-latency classification. Experiments with 12 participants across three affective states demonstrate that TED-Net achieves 85.98% accuracy, outperforming 16 baseline methods. Ablation studies confirm that tri-modality fusion yields 16.38% improvement over the best single-modality baseline and 5.29% improvement over the best dual-modality combination, while t-SNE visualization validates the discriminative power of learned multimodal representations. The proposed system offers a practical solution for real-time emotion monitoring in mobile computing environments. The code is available online at GitHub.