Self-adaptive Perception Model for Action Segment Detection

Action segment detection is an important yet challenging problem, since we need to localize the proposals which contain an action instance in a long untrimmed video with arbitrary length and random position. This task requires us not only to find the precise moment of starting and ending of an action instance, but also to detect action instances as many as possible. We propose a new model Self-Adaptive Perception to address this problem. We predict the action boundaries by classifying start and end of each action as separate components, allowing our model to predict the starting and ending boundaries roughly and generate candidate proposals. We evaluate each candidate proposals by a novel and flexible architecture called Discriminator. It can extract enough semantic information and generate precise confidence score of whether a proposal contains an action within its region, which benefit from the self-adaptive architecture. We conduct solid and rich experiments on large dataset Activity-Net, the result shows that our method achieves a competitive performance, outperforming most published state-of-the-art method in the field. And further experiments demonstrate the effect of each module of our model.