A Deep Learning Network for Action Recognition Incorporating Temporal Attention Mechanism

Yue Liu, Lei Zhang*, Shan Xin, Yu Zhang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Although motion recognition is widely used in various research fields, the performance of traditional motion recognition methods is poor in complex environments. In this paper a method for pedestrian action recognition in complex environments is proposed. A network for action recognition incorporating temporal attention mechanism is proposed. The main improvement of the method is as follows: firstly, RCNN network is used for pedestrian detection to get the locations of all pedestrians in videos. Secondly, long and short term memory network (LSTM) is used to extract temporal features. On one hand, the network uses a residual part incorporating a spatial attention mechanism to extract the spatial features, which could reduce the interference from the image background. On the other hand, the Temporal Attention Mechanism (TAM) is introduced, which dynamically allocates video frame sequence weights according to the importance of LSTM output. Finally, experiments are conducted on the UCF101 dataset to verify the improvement of the accuracy and precision of the method.

Original languageEnglish
Title of host publication2021 IEEE International Conference on Robotics and Biomimetics, ROBIO 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1576-1581
Number of pages6
ISBN (Electronic)9781665405355
DOIs
Publication statusPublished - 2021
Externally publishedYes
Event2021 IEEE International Conference on Robotics and Biomimetics, ROBIO 2021 - Sanya, China
Duration: 27 Dec 202131 Dec 2021

Publication series

Name2021 IEEE International Conference on Robotics and Biomimetics, ROBIO 2021

Conference

Conference2021 IEEE International Conference on Robotics and Biomimetics, ROBIO 2021
Country/TerritoryChina
CitySanya
Period27/12/2131/12/21

Keywords

  • Action Recognition
  • Attention Mechanism
  • LSTM
  • ResNet50

Fingerprint

Dive into the research topics of 'A Deep Learning Network for Action Recognition Incorporating Temporal Attention Mechanism'. Together they form a unique fingerprint.

Cite this