Video based emotion recognition using CNN and BRNN

Youyi Cai, Wenming Zheng*, Tong Zhang, Qiang Li, Zhen Cui, Jiayin Ye

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

17 Citations (Scopus)

Abstract

Video-based Emotion recognition is a rather challenging computer vision task. It not only needs to model spatial information of each image frame, but also requires considering temporal contextual correlations among sequential frames. For this purpose, we propose a hierarchical deep network architecture to extract high-level spatial-temporal features. In this architecture, two classic deep neural networks, convolutional neutral networks (CNN) and bi-directional recurrent neutral networks (BRNN), are employed to respectively capture facial textural characteristics in spatial domain and dynamic emotion changes in temporal domain. We endeavor to coordinate the two networks by optimizing each of them, so as to boost the performance of the emotion recognition. In the challenging competition, our method achieves a promising performance compared with the baselines.

Original languageEnglish
Title of host publicationPattern Recognition - 7th Chinese Conference, CCPR 2016, Proceedings
EditorsTieniu Tan, Xilin Chen, Xuelong Li, Jian Yang, Hong Cheng, Jie Zhou
PublisherSpringer Verlag
Pages679-691
Number of pages13
ISBN (Print)9789811030048
DOIs
Publication statusPublished - 2016
Externally publishedYes

Publication series

NameCommunications in Computer and Information Science
Volume663
ISSN (Print)1865-0929

Keywords

  • Bi-directional recurrent neutral networks (BRNN)
  • Convolutional neutral networks (CNN)
  • Emotion recognition

Fingerprint

Dive into the research topics of 'Video based emotion recognition using CNN and BRNN'. Together they form a unique fingerprint.

Cite this