FARP-Net: Local-Global Feature Aggregation and Relation-Aware Proposals for 3D Object Detection

Tao Xie, Li Wang, Ke Wang*, Ruifeng Li*, Xinyu Zhang*, Haoming Zhang, Linqi Yang, Huaping Liu, Jun Li

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

11 Citations (Scopus)

Abstract

In this work, we introduce FARP-Net, an adaptive local-global feature aggregation and relation-aware proposal network for high-quality 3D object detection from pure point clouds. Our key insight is that learning adaptive local-global feature aggregation from an irregular yet sparse point cloud and generating superb proposals are both pivotal for detection. Technically, we propose a novel local-global feature aggregation layer (LGFAL) that fully exploits the complementary correlation between local features and global features, and fuses their strengths adaptively via an attention-based fusion module. Furthermore, we incorporate a lightweight feature affine module (LFAM) into LGFAL to map the local features into a normal distribution, thus acquiring fine-grained features of each local region in a weight-sharing manner. During object proposal generation, we propose a weighted relation-aware proposal module (WRPM) that uses an objectness-aware formalism to weigh the relation importance among object candidates for a clear and principal context, thereby facilitating the generation of high-quality proposals. The WRPM challenges the traditional practice of extracting contextual information among all object candidates, which is inefficient as object candidates are always noisy and redundant. Experimentally, FARP-Net delivers superior performance on two widely used benchmarks with fewer parameters, 64.0% mAP@0.25 on the SUN RGB-D dataset and 70.9% mAP@0.25 on the ScanNet V2 dataset. We further validate that the proposed LGFAL and WRPM can be integrated into both indoor and outdoor detectors to boost performance.

Original languageEnglish
Pages (from-to)1027-1040
Number of pages14
JournalIEEE Transactions on Multimedia
Volume26
DOIs
Publication statusPublished - 2024
Externally publishedYes

Keywords

  • 3D object detection
  • deep learning
  • point cloud

Fingerprint

Dive into the research topics of 'FARP-Net: Local-Global Feature Aggregation and Relation-Aware Proposals for 3D Object Detection'. Together they form a unique fingerprint.

Cite this