PixelFusion: A Pixel-level Multi-Sensor Fusion Method for 3D Object Detection

Qili Ning, Qianfei Liu, Zhang Zhang, Haoyu Li, Zitong Chen, Chao Sun*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Multi-sensor fusion stands as a pivotal approach for enhancing the efficacy of 3D object detection in the context of autonomous driving. Nowadays, the fusion methods are mainly based on BEV (Bird's-Eye View). This kind of methods benefits from the advantage of BEV features fitting geometry size and distance, but they are limited by BEV's weakening of camera semantics features and limited detection range. In view of this situation, we propose a 3D object detection framework based on visual stream, dubbed PixelFusion, which unifies the point clouds and semantic images into the pixel-level unit of the image. It avoids the limitation of BEV-based methods, which often discard the semantics density of camera features and are constrained by the BEV range. In order to achieve a better fusion result, we also introduce a Dynamic Feature Selection module designed to optimize the integration of image and point cloud data. Our experimental results show that, after training, our framework outperforms several classical 3D object detection methods. To the best of our knowledge, this is the first method to convert Lidar point clouds into 3-channel pixel-level images.

Original languageEnglish
Title of host publicationProceedings of the 2024 8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798331504892
DOIs
Publication statusPublished - 2024
Event8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024 - Chongqing, China
Duration: 25 Oct 202427 Oct 2024

Publication series

NameProceedings of the 2024 8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024

Conference

Conference8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024
Country/TerritoryChina
CityChongqing
Period25/10/2427/10/24

Keywords

  • Autonomous Vehicle
  • Multi-modality
  • Object Detection
  • Pixel-level Fusion

Cite this