Denoised Temporal Relation Network for Temporal Action Segmentation

Temporal relations among action segments play a crucial role in temporal action segmentation. Existing methods tend to employ the graph neural network to model the temporal relation. However, the performance is unsatisfactory and exhibits serious over-segmentation due to the generated noisy features. To solve the above issues, we present an action segmentation framework, termed a denoised temporal relation network (DTRN). In DTRN, a temporal reasoning module (TRM) models inter-segment temporal relations and conducts feature denoising jointly. Specifically, the TRM conducts an uncertainty-gated reasoning mechanism for noise-immune and utilizes a cross-attention-based structure to combine the informative clues from the discriminative enhance module which is trained under Selective Margin Plasticity (SMP) to ensure informative clues, SMP adjusts the decision boundary adaptively by changing specific margins in real-time. Our framework is demonstrated to be effective and achieves state-of-the-art performance of accuracy, edit score, and F1 score on the challenging 50Salads, GTEA, and Breakfast benchmarks.