A Cycle Architecture Based on Policy Gradient for Unsupervised Video Summarization

This paper proposes a cycle architecture based on policy gradients for unsupervised video summarization. Specifically, the Modified DSNet and DSN-attention net constitute a cycle architecture and promote each other in the training stage to achieve higher performance compared with the unsupervised methods that formulate video summarization as a sequential decision-making process. In the training stage, the DSN-attention net is trained by the policy gradient in combination with the additional MSE loss between the two outputs of the modified DSNet and DSN-attention net. Then the output of the DSN-attention net is taken for generating the labels to train the modified DSNet. As a result, a cycle architecture is built up for unsupervised video summarization. At the test stage, the final video summary is produced by the average fusion of the outputs of both the Modified DSNet and DSN attention net. Extensive experiments and analysis on two benchmark datasets demonstrate the effectiveness of our method and its superior performance in comparison with the state-of-the-art unsupervised methods.