A Deep Learning Network for Action Recognition Incorporating Temporal Attention Mechanism

Although motion recognition is widely used in various research fields, the performance of traditional motion recognition methods is poor in complex environments. In this paper a method for pedestrian action recognition in complex environments is proposed. A network for action recognition incorporating temporal attention mechanism is proposed. The main improvement of the method is as follows: firstly, RCNN network is used for pedestrian detection to get the locations of all pedestrians in videos. Secondly, long and short term memory network (LSTM) is used to extract temporal features. On one hand, the network uses a residual part incorporating a spatial attention mechanism to extract the spatial features, which could reduce the interference from the image background. On the other hand, the Temporal Attention Mechanism (TAM) is introduced, which dynamically allocates video frame sequence weights according to the importance of LSTM output. Finally, experiments are conducted on the UCF101 dataset to verify the improvement of the accuracy and precision of the method.