STU-Mamba: A U-Mamba With Cross-Scale Alignment and Temporal Adaptive Context Gating for Satellite Image Time Series Segmentation

Semantic segmentation of Satellite Image Time Series (SITS) is essential for monitoring Earth's surface dynamics, particularly in agricultural applications. However, existing methods often suffer from spatial misalignment due to downsampling and show limited adaptability to the spatiotemporal heterogeneity of crop phenology. To address this, we propose STU-Mamba, a novel architecture that integrates selective state space models (SSMs) with a U-shaped framework. For spatial modeling, we introduce a cross-scale aligned encoding mechanism that combines a Cross-Scale Alignment (CSA) module and Dual-Directional Mamba (DDM) blocks. This design preserves spatial details, mitigates geometric misalignment during encoding, and captures omnidirectional contextual dependencies. For temporal modeling, we present a Temporal Adaptive Context Gating (TACG) module. Within it, a Mamba block encodes long-range dependencies, and a context-aware gating network generates pixel-wise weights based on local spatial context to collapse the temporal dimension via adaptive aggregation. This enables the model to accurately characterize the phenological dynamics of the same crop type across different regions, effectively addressing the spatiotemporal heterogeneity inherent in agricultural landscapes. Finally, we construct a unified encoder-decoder framework entirely driven by Mamba operators, also embedding Mamba in the decoder for spatial refinement. Experiments on two public benchmarks, MTLCC and PASTIS, show that STU-Mamba achieves state-of-the-art performance, reaching 79.10% mean Intersection over Union (mIoU) on MTLCC and outperforming previous methods. Our approach delivers high accuracy and favorable scaling with respect to sequence length, making it suitable for large-scale SITS analysis.