K-means Clustering-based Kernel Canonical Correlation Analysis for Multimodal Emotion Recognition in Human-Robot Interaction

In this paper, K-means Clustering-based Kernel Canonical Correlation Analysis (KMKCCA) algorithm is proposed for multimodal emotion recognition in Human-Robot Interaction (HRI). The multimodal features (Gray pixels, time domain and frequency domain) extracted from facial expression and speech are fused based on Kernel Canonical Correlation Analysis. K-means clustering is used to select features from multiple modalities and reduces dimensionality. The proposed approach can improve the heterogenicity among different modalities and make multiple modalities complementary to promote multimodal emotion recognition. Experiments on two datasets, namely SAVEE and eNTERFACE05, are conducted to evaluate the accuracy of the proposed method. The results show that the proposed method produces good recognition rates that are higher than the ones produced by the methods without K-means Clustering, more specifically, they are 2.77% higher in SAVEE and 4.7% higher in eNTERFACE05.