Multi-Convolution Neural Networks-Based Deep Learning Model for Emotion Understanding

Multi-convolution neural networks-based deep learning model in combination with multimodal data for emotion understanding is proposed, in which facial expression and body gesture are used to achieve emotional states recognition for emotion understanding. It aims to understand coexistence multimodal information in human-robot interaction by using multi-convolution neural networks, where multilayer convolutions are connected in series and multiple networks are executed in parallel. Moreover, when optimizing the weights of deep neural network by traditional method, it is easy to fall into poor local optimal. To address this problem, a hybrid genetic algorithm with stochastic gradient descent is developed, which has the capacity of inherent implicit parallelism and better global optimization of genetic algorithm so that it can adaptively find the better weights of the network. And in order to speed up the convergence of the proposal, the weights optimized by stochastic gradient descent will be taken as a chromosome of genetic algorithms initial population, and it also can be used as a priori knowledge. To verify the effectiveness of the proposal, experiments on benchmark database of spontaneous emotion expressions are developed, and experimental results show that the proposal outperforms the state-of-the-art methods. Meanwhile, the preliminary application experiments are also carried out and the results indicate that the proposal can be extended to human-robot interaction.