Action Recognition Based on a Hybrid Deep Network

To get higher accuracy of the CNN action recognition networks with video inputs, many methods will deepen or modify the convolutional layers of the original networks. However, this would result in a substantial increase of the parameters and cost resources. In this paper, we propose an efficient and versatile method with good transfer performance to further improve the accuracy of action recognition networks while avoiding the increase of network volume. It combines 3D convolutional neural network (CNN) and online-sequential extreme learning machine (OS-ELM) as a hybrid architecture and makes the most of their advantages while overcoming their primary drawbacks: The outstanding feature extraction ability of 3D CNN is fully utilized, while the deficiencies of 3D CNN in classification and network parameters are remedied by the OS-ELM, which has better ability of generalization but has difficulties in feature extraction tasks. Besides, we not only give the details of our structure including the design and analysis but also investigate the influence of different pre-training datasets on training and validation stages respectively. Through experiments on two datasets UCF101 and HMDB51, we find that the model with our proposed hybrid structure provides better recognition precision in action recognition with fewer computation parameters, which verifies the effectiveness of our proposed structure and validates that it can be used for future improvement to recognition models.