MDTNet: Multiscale Deformable Transformer Network With Fourier Space Losses Toward Fine-Scale Spatiotemporal Precipitation Nowcasting

Deep learning (DL)-based precipitation nowcasting algorithms have garnered significant attention in recent years. However, the presence of variable spatial scales in precipitation patterns poses challenges for methods that solely focus on capturing spatiotemporal correlations at a single scale. Moreover, current DL-based algorithms tend to model short-term (e.g., 10-min time span) rainfall locally neglecting long-term, global (e.g., 2-h time span) life-cycle evolution. Furthermore, widely used pixel-wise losses are prone to produce low effective-spatial-resolution predictions. To this end, we introduce a multiscale deformable transformer network to leverage echo contexts from image patches of varying spatial scales. Meanwhile, a multihead deformable self-attention mechanism is introduced for capturing precipitation spatiotemporal dynamics in a global manner. Moreover, to improve the spatial resolution of predictions, the Fourier space regularization and adversarial losses are proposed by narrowing the discrepancy of the Fourier spectra of predictions and references. Thanks to the introduced loss function, our model generates highly effective spatial-resolution predictions with abundant details. Extensive experiments on two real datasets show the substantial superiority of our method in terms of critical success index (CSI) compared to recent competitive approaches. At the same time, our predictions have more realistic precipitation details and significantly better fidelity. For example, on a vertically integrated liquid (VIL) product dataset, compared to baseline methods, our approach reduces the Fréchet inception distance (FID) value by a factor of 2sim 4 while improves the CSI score by 3%~5% approximately.