CCRDet: Cross-modal complementary region-aware framework for drone-based RGBT tiny person detection

Drone-based RGBT person detection, which has garnered significant attention, seamlessly integrates the spatial flexibility of drones with the around-the-clock time flexibility of RGBT data for continuous information acquisition. However, the drone-captured images cover a wide range of areas, resulting in illumination imbalance and thermal background clutter issues in RGB and thermal images. The varying quality of the two modalities in different regions of an RGBT image set increases the difficulty of complementary information fusion, thereby leading to false negatives in the advanced drone-based RGBT tiny person detection methods. In this context, a novel Cross-modal Complementary Region-aware framework for drone-based RGBT tiny person Detection (CCRDet) is proposed for effective object detection under poorly illuminated RGB regions and thermally cluttered backgrounds. CCRDet employs the proposed cross-modal region-aware guidance to be aware of these regions and guide the counterpart modality to enhance valid target features accordingly. After that, it leverages the proposed modality-difference feature gated fusion to deliver these valid target features to the fused features with effective preservation, thereby enhancing their response intensity after fusion and providing high-quality inputs for the detection head. Extensive experiments on two drone-based RGBT tiny person detection datasets, RGBTDronePerson and VTUAV-det, demonstrate the effectiveness of the proposed method. The code is available at https://github.com/G-pz/CCRDet .