Breaking the alignment barrier: A spatiotemporal alignment-free RGBT tracking approach

RGB-Thermal (RGBT) tracking leverages complementary information from visible and infrared modalities to improve tracking robustness in complex environments. However, its practical deployment remains constrained by the stringent requirement for precise spatiotemporal alignment between heterogeneous modalities—a condition rarely satisfied in real-world applications. To overcome this limitation, we present SAFNet, a novel Spatiotemporal Alignment-Free Network that eliminates the need for precise cross-modal alignment through innovative architectural designs. Our framework develops a spatiotemporal interaction query module incorporating cross-modal temporal attention, which re-establishes inter-modal temporal correlations for unregistered inputs by leveraging similarity learning across asynchronous data streams. For spatial discrepancy mitigation, we propose a dual-branch pre-tracking network employing deep cross-correlation analysis, combined with an adaptive feature fusion strategy under the guidance of joint response distribution. Furthermore, we devise an innovative dynamic template update mechanism that adaptively adjusts modal update rates to maintain temporal consistency across heterogeneous data streams. Comprehensive evaluations validate SAFNet's state-of-the-art performance across four benchmark datasets (GTOT, RGBT210, RGBT234, LasHeR), demonstrating significant improvements in tracking accuracy. The proposed architecture represents a significant advancement toward practical deployment of robust RGBT tracking systems in real-world environments with asynchronous multimodal inputs.