Intelligent Vehicle Multi-Sensor Data Fusion using Deep Bipartite Graph Matching

Sensor fusion proves to be an effective method for enhancing the perception capabilities of intelligent vehicles. Traditional matching methods, such as bipartite graph matching, struggle to achieve high precision and robustness in aligning data from LiDAR, radar, and camera sensors. To address this, our proposed solution introduces a deep bipartite graph matching (DBGM) algorithm for the fusion of multi-sensor data in intelligent vehicles. Initially, the perceptual outputs from these sensors are transformed into occupancy feature maps within the camera's pixel plane. Subsequently, a Transformer-based self-attention module is employed for feature extraction. The results obtained from the bipartite graph matching supervise the self-attention module and serve as regression training labels for the fully connected layers (FC) module. In experiments conducted on the NuScenes dataset, our algorithm demonstrates a 7.4% improvement in Fl score compared to conventional methods. It also showcases exceptional perception accuracy and robustness across a variety of real-world driving scenarios. This algorithm presents an efficient and precise solution for the fusion of multi-sensor data in intelligent vehicles.