A Temporal Action Detection Model Based on Deep Reinforcement Learning

Existing methods for temporal action detection typically follow a two-stage approach, generating numerous action proposals at various scales on untrimmed long videos. These proposals are then used with a pre-trained action classifier to estimate the action type and accuracy for each proposal. This method not only generates a large number of irrelevant proposals, leading to significant computational overhead, but also deviates from the human perception process. This paper solves the temporal action detection problem from a human cognition perspective, considering it as a process of observing and refining proposals to locate temporal actions. The paper introduces a deep reinforcement learning-based Temporal Action Detection model which leverages deep learning networks to understand the temporal information in video sequences. Through reinforcement learning, the agent learns strategies to change the position of proposals, aiming to locate different temporal actions. The Action Buffer module records the actions performed by the proposals, the Proposal Regression Network refines the position deviation between the predicted results and labels, ensuring more accurate results. Correct proposals are stored in the Proposal Buffer module to avoid redundant predictions. Experimental results on the THUMOS'14 dataset confirm the accuracy of the model, achieving excellent results in metrics such as AP values and recall.