MSTMNet: A Multi-Scale Temporal Memory Network with Dynamic Memory Bank for Video Salient Object Detection

In dynamic visual scenes, salient object detection faces critical challenges in maintaining temporal consistency and spatial accuracy, particularly manifested in insufficient spatiotemporal feature integration and inadequate modeling of long-term temporal dependencies. To tackle these challenges, we introduce the Multi-Scale Temporal Memory Network (MSTMNet), a novel framework that synergizes hierarchical spatiotemporal modeling with adaptive memory mechanisms. MSTMNet employs a dual-encoder architecture to separately extract features from the current query frame and historical memory frames, ensuring rich contextual representation. A dynamic memory bank with an attention-guided elimination policy selectively retains historically significant features based on relevance and freshness, while a Multi-Temporal Feature Fusion (MTFF) module aggregates multi-scale features across sequential temporal steps to capture both short-term motion cues and long-range dependencies. Subsequently, within the Spatial Feature Fusion (SFF) module, an enhanced self-attention mechanism refines low-level features by propagating semantic consistency from high-level representations. By combining memory-enhanced features with query features through a hierarchical decoder, MSTMNet achieves precise saliency maps. Comprehensive experiments on public datasets validate that MSTMNet surpasses transformer-based methods and outperforms established semi-supervised and optical-flow-dependent techniques in accuracy.