Multispectral Feature-Fusion Object Detection Based on Multi-Scale Auxiliary Branch

Object detection in real-world environments is inherently challenging due to various factors, such as poor illumination conditions, which can significantly hinder detection accuracy. To overcome the above challenges, we propose a multispectral feature-fusion object detection framework by designing a dual-modal backbone based on YOLOv8. Within the backbone, we propose a Feature Reconstruction Fusion module based on the Swin Transformer, which reconstructs dual-modal features by considering the intensity of spatial information and learns complementary inter-modal information. Moreover, we design multi-scale auxiliary branches during the training stage to complement the single-modal gradient information at different levels, strengthening the model's ability to recognize multi-scale targets. Our method is tested on several public datasets, including the FLIR-aligned, LLVIP, and M³FD datasets. The results show that our network not only achieves remarkable accuracy in detecting small-sized targets but also outperforms other state-of-the-art networks in mean average precision.