Video annotation by using heterogeneous multiple image groups on the Web

Annotating events in uncontrolled videos is a challenging task. Current researches mainly focus on obtaining concepts from numerous labeled videos. But it is time consuming and labor expensive to collect a large amount of required labeled videos to model events under various circumstances. To alleviate the labeling process, we propose to learn models for videos by leveraging abundant Web images which cover many roughly annotated events under various conditions. However, knowledge from the Web is noisy and diverse, brute force knowledge transfer may hurt the annotation performance. To address such negative transfer problem, we propose a novel Joint Group Weighting Learning (JGWL) framework to leverage different but related groups of knowledge (source domain) learned from the Web images to real-world videos (target domain). Under this framework, weights of different groups are learned in a joint optimization problem, and each weight represents how contributive the corresponding image group is to the knowledge transferred to the video. Moreover, to deal with the problem of heterogeneous feature spaces between videos and images, we build a common feature subspace to bridge image and video feature spaces. Experimental results on two challenging video datasets demonstrate that it is effective to use grouped knowledge gained from Web images for video annotation.