TY - GEN
T1 - MMOT
T2 - 39th Conference on Neural Information Processing Systems, NeurIPS 2025
AU - Li, Tianhao
AU - Xu, Tingfa
AU - Wang, Ying
AU - Qin, Haolin
AU - Lin, Xu
AU - Li, Jianan
N1 - Publisher Copyright:
© 2025 Neural information processing systems foundation. All rights reserved.
PY - 2025
Y1 - 2025
N2 - Drone-based multi-object tracking is essential yet highly challenging due to small targets, severe occlusions, and cluttered backgrounds. Existing RGB-based multi-object tracking algorithms heavily depend on spatial appearance cues such as color and texture, which often degrade in aerial views, compromising tracking reliability. Multispectral imagery, capturing pixel-level spectral reflectance, provides crucial spectral cues that significantly enhance object discriminability under degraded spatial conditions. However, the lack of dedicated multispectral UAV datasets has hindered progress in this domain. To bridge this gap, we introduce MMOT, the first challenging benchmark for drone-based multispectral multi-object tracking dataset. It features three key characteristics: (i) Large Scale - 125 video sequences with over 488.8K annotations across eight object categories; (ii) Comprehensive Challenges - covering diverse real-world challenges such as extreme small targets, high-density scenarios, severe occlusions, and complex platform motion; and (iii) Precise Oriented Annotations - enabling accurate localization and reduced object ambiguity under aerial perspectives. To better extract spectral features and leverage oriented annotations, we further present a multispectral and orientation-aware MOT scheme adapting existing MOT methods, featuring: (i) a lightweight Spectral 3D-Stem integrating spectral features while preserving compatibility with RGB pretraining; (ii) an orientation-aware Kalman filter for precise state estimation; and (iii) an end-to-end orientation-adaptive transformer architecture. Extensive experiments across representative trackers consistently show that multispectral input markedly improves tracking performance over RGB baselines, particularly for small and densely packed objects. We believe our work will benefit the community in advancing drone-based multispectral multi-object tracking research. Our MMOT, code and benchmarks are publicly available at https://github.com/Annzstbl/MMOT.
AB - Drone-based multi-object tracking is essential yet highly challenging due to small targets, severe occlusions, and cluttered backgrounds. Existing RGB-based multi-object tracking algorithms heavily depend on spatial appearance cues such as color and texture, which often degrade in aerial views, compromising tracking reliability. Multispectral imagery, capturing pixel-level spectral reflectance, provides crucial spectral cues that significantly enhance object discriminability under degraded spatial conditions. However, the lack of dedicated multispectral UAV datasets has hindered progress in this domain. To bridge this gap, we introduce MMOT, the first challenging benchmark for drone-based multispectral multi-object tracking dataset. It features three key characteristics: (i) Large Scale - 125 video sequences with over 488.8K annotations across eight object categories; (ii) Comprehensive Challenges - covering diverse real-world challenges such as extreme small targets, high-density scenarios, severe occlusions, and complex platform motion; and (iii) Precise Oriented Annotations - enabling accurate localization and reduced object ambiguity under aerial perspectives. To better extract spectral features and leverage oriented annotations, we further present a multispectral and orientation-aware MOT scheme adapting existing MOT methods, featuring: (i) a lightweight Spectral 3D-Stem integrating spectral features while preserving compatibility with RGB pretraining; (ii) an orientation-aware Kalman filter for precise state estimation; and (iii) an end-to-end orientation-adaptive transformer architecture. Extensive experiments across representative trackers consistently show that multispectral input markedly improves tracking performance over RGB baselines, particularly for small and densely packed objects. We believe our work will benefit the community in advancing drone-based multispectral multi-object tracking research. Our MMOT, code and benchmarks are publicly available at https://github.com/Annzstbl/MMOT.
UR - https://www.scopus.com/pages/publications/105048580445
M3 - Conference contribution
AN - SCOPUS:105048580445
T3 - Advances in Neural Information Processing Systems
SP - 161692
EP - 161721
BT - 39th Conference on Neural Information Processing Systems, NeurIPS 2025
PB - Neural information processing systems foundation
Y2 - 2 December 2025 through 7 December 2025
ER -