Emotion Recognition Based on Multi-View Body Gestures

Zhijuan Shen, Jun Cheng*, Xiping Hu, Qian Dong

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

33 Citations (Scopus)

Abstract

Body gesture, a crucial component of 'body language', remains less explored to recognize emotion while face expression-based and speech-based approaches are widely investigated. In this paper, we introduce an exploratory experiment to recognize emotion using deep learning only from body gestures. 43,200 multi-view RGB videos of simplified body gestures and their neutral control groups are captured from 80 humans using Hikvision network cameras to support the experiment. A novel approach is proposed to use deep neural network fuse skeleton and RGB features only using single-modality RGB video data. Experimental results show our approach achieves substantial improvements both in individual categories and overall and is provided with stronger generalization capability as well.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
PublisherIEEE Computer Society
Pages3317-3321
Number of pages5
ISBN (Electronic)9781538662496
DOIs
Publication statusPublished - Sept 2019
Externally publishedYes
Event26th IEEE International Conference on Image Processing, ICIP 2019 - Taipei, Taiwan, Province of China
Duration: 22 Sept 201925 Sept 2019

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2019-September
ISSN (Print)1522-4880

Conference

Conference26th IEEE International Conference on Image Processing, ICIP 2019
Country/TerritoryTaiwan, Province of China
CityTaipei
Period22/09/1925/09/19

Keywords

  • body gesture
  • emotion recognition
  • feature fusion
  • multi-view

Fingerprint

Dive into the research topics of 'Emotion Recognition Based on Multi-View Body Gestures'. Together they form a unique fingerprint.

Cite this