Dynamic token halting for efficient abstractive summarization with importance-aware regularization

Text summarization involves distilling key information from a lengthy document and presenting it as a concise summary. Unlike extractive summarization, which directly selects phrases or sentences from the source text, abstractive summarization requires comprehending the entire document and generating a summary word by word. Current state-of-the-art abstractive summarization systems rely on large pretrained models, which often suffer from inefficiencies caused by overprocessing irrelevant information. A significant portion of unimportant data is unnecessarily encoded, leading to excessive computational costs. In this paper, we address these inefficiencies by introducing a method that incrementally discards redundant hidden states throughout the encoding process, achieved by leveraging the Adaptive Computation Time (ACT) mechanism. Additionally, we propose a novel Importance-aware Prior Regularization technique that helps the model identify and prioritize crucial parts of the document, ensuring they are processed more thoroughly in deeper layers of the encoder. Our approach reduces the computational demands of pretrained encoders by 25%–35% in terms of floating-point operations (FLOPs) while maintaining performance levels comparable to strong baseline models. Extensive experiments demonstrate that our method is particularly effective at reducing computational costs for longer documents. The code and data will be made publicly available upon acceptance of this paper.