Tracking and Identification of Weakly Correlated Drones Swarm Based on Multimodal Information Fusion of Radar and Camera

In the field of drone detection, existing multi-sensor information fusion methods have been primarily focused on individual or sparsely distributed targets, without taking into account the characteristics of swarms. There is currently a lack of publicly available datasets for multi-sensor drone swarm detection, and research on swarm detection is scarce, with studies mainly focusing on single-sensor approaches. A novel multimodal information fusion method combining radar and camera data has been introduced. This approach utilizes visual Multi-Object Tracking (MOT) to extract classes and IDs from images and acquires implicit angular information through spatiotemporal registration. These pieces of information, alongside radar measurements, are used in the processes of prediction, filtering, matching, fusion, and association to determine the trajectories of each drone in the swarm and to identify their types. Comparisons with other methods have demonstrated that this fusion approach significantly improves the accuracy and speed of tracking drone swarms, highlighting its potential to enhance drone detection and tracking capabilities in complex scenarios.