Visible and Infrared Image Object Detection Based on FPGA

Currently, most object detection systems based on embedded are single-modal, which results in relatively low detection accuracy. In contrast, dual-modal object detection systems offer higher accuracy but come with increased hardware resource requirements and higher power consumption. By employing a lightweight dual-modal object detection network for RGB and infrared images and a parallel-pipeline reconfigurable hardware structure based on block convolution, this paper designs a high-performance object detection system based on FPGA. Compared to the original network model, the lightweight processed model after undergoing techniques such as pruning, sparsity and quantization, demonstrates a significant reduction of parameters to 24% of the original, with only a 1.1% drop in mAP@0.5. Based on the Milianke F25 development board equipped with a Xilinx xczu15eg chip, the system achieves a detection speed of 29.3ms, with an energy efficiency improvement of approximately 4.26 times compared to the NVIDIA GeForce RTX 3060.