RGBT Decision Stage Fusion Perception Based on Improved YOLOv8

To enhance the perception capabilities of autonomous vehicles in complex environments, we propose a Visible Light-Infrared (RGBT) decision stage fusion perception method. Based on the YOLOv8 model, a Bidirectional Feature Pyramid Network (BiFPN) structure is introduced, along with the integration of the Convolutional Block Attention Module (CBAM) at three different points for comparative analysis. This optimized model leads to improved detection accuracy. Subsequently, the Non-Maximum Suppression (NMS) algorithm is employed to integrate detection results from both modalities. Experimental results demonstrate that the improved model achieves heightened recognition accuracy on the FLIR dataset for both visible light and infrared scenes, with increases in mAP50 of 1.4% and 2.2% respectively. The enhancement in Fusion perception in mAP50 improved by 8.6% compared to using only visible-light perception., confirming the effectiveness of the visible light-infrared scene perception in challenging environments.