MFMG-Net: Multispectral Feature Mutual Guidance Network for Visible–Infrared Object Detection

Fei Zhao, Wenzhong Lou*, Hengzhen Feng*, Nanxi Ding, Chenglong Li

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Drones equipped with visible and infrared sensors play a vital role in urban road supervision. However, conventional methods using RGB-IR image pairs often struggle to extract effective features. These methods treat these spectra independently, missing the potential benefits of their interaction and complementary information. To address these challenges, we designed the Multispectral Feature Mutual Guidance Network (MFMG-Net). To prevent learning bias between spectra, we have developed a Data Augmentation (DA) technique based on the mask strategy. The MFMG module is embedded between two backbone networks, promoting the exchange of feature information between spectra to enhance extraction. We also designed a Dual-Branch Feature Fusion (DBFF) module based on attention mechanisms, enabling deep feature fusion by emphasizing correlations between the two spectra in both the feature channel and space dimensions. Finally, the fused features feed into the neck network and detection head, yielding ultimate inference results. Our experiments, conducted on the Aerial Imagery (VEDAI) dataset and two other public datasets (M3FD and LLVIP), showcase the superior performance of our method and the effectiveness of MFMG in enhancing multispectral feature extraction for drone ground detection.

Original languageEnglish
Article number112
JournalDrones
Volume8
Issue number3
DOIs
Publication statusPublished - Mar 2024

Keywords

  • RGB-IR image pair
  • attention mechanism
  • multispectral feature
  • object detection

Fingerprint

Dive into the research topics of 'MFMG-Net: Multispectral Feature Mutual Guidance Network for Visible–Infrared Object Detection'. Together they form a unique fingerprint.

Cite this