Hybrid Source Selection Fusion Domain-invariant Attention for Cross-subject Emotion Recognition

Electroencephalogram (EEG) has been widely used for emotion recognition due to its portability and high temporal resolution. It makes success in subject-dependent scenario but faces significant challenges in cross-subject emotion recognition because of non-stationarity of EEG and individual differences. Most previous studies treat all individuals as a single source domain for transferring emotional knowledge, which may introduce irrelevant information and lead to negative transfer. Besides, there is a potential risk that some important information of common emotional features might be ignored. To deal with the issues, we propose a framework called hybrid source selection fusion domain-invariant attention (HSSFDA) for cross-subject emotion recognition. First, source domains are selected by leveraging local and global similarity for knowledge transfer. Then, a specialized attention mechanism is employed to focus on important emotional information extracted from the domain-invariant features. Finally, domain-invariant and domain-specific features are fused to enhance emotion recognition performance. To evaluate the proposed method, experiments are conducted on several public datasets including SEED, SEED_IV, DREAMER and DEAP. The results demonstrate that HSSFDA achieves accuracies of 85.07 %, 72.11 %, 62.36 %, 77.17 %, 58.51 %, and 63.55 % on SEED, SEED_IV, valence and arousal of DREAMER, and valence and arousal of DEAP datasets, respectively, demonstrating competitive performance compared to popular and state-of-the-art methods. Furthermore, we apply the HSSFDA to a self-recorded dataset collected by self-developed three-channel device and validate its effectiveness in practical applications. In conclusion, HSSFDA is a feasible method for cross-subject emotion recognition and has the potential to broaden the application of EEG in the field of affective computing.