Action recognition with motion map 3D network

Recently, deep neural networks have demonstrated remarkable progresses for human action recognition in videos. However, most existing deep frameworks can not handle variable-length videos properly, which leads to the degradation in classification performance. In this paper, we propose a Motion Map 3D ConvNet(MM3D), which can represent the content of a video with arbitrary video length by a motion map. In our MM3D model, a novel generation network is proposed to learn a motion map to represent a video clip by iteratively integrating a current video frame into a previous motion map. A discrimination network is also introduced for classifying actions based on the learned motion map. Experiments on the UCF101 and the HMDB51 datasets prove the effectiveness of our method for human action recognition.