A rule-based hierarchical framework for video parsing in compressed domain

In this paper, a rule-based hierarchical approach is proposed for video parsing in the compressed domain. In this approach, three core factors of video parsing that is feature space, discontinuity metric and decision rule are stressed to improve the parsing performance. Firstly, the PCA-based feature is extracted in the compressed domain as the input of the framework, which shows superior performance than the traditional features, especially in the compressed domain. Secondly, an adaptive temporal multi-scale discontinuity metric is designed to measure the temporal inconsistency of the visual feature between two video frames. Finally, through well characterizing the local activity of the shot boundary, a reasonable decision rules based on the adaptive threshold is proposed to parse the video into shots. Through this PCA-based framework, the negative influence of significant camera or objects movement to the video parsing can be greatly depressed, and thus the performance is greatly improved. The proposed approach is implemented on the TREC video test repository to validate the performance.