PUDet: Advancing 3D Object Detection with Generative Upsampling Networks

Lidar-based 3D object detection achieves superior performance. However, the unevenly distributed point clouds on foreground objects can weaken their geometric representation. Moreover, far-away objects typically have very few points, which further impairs detection performance. In this paper, we present a novel framework PUDet (Point Cloud Upsampling 3D Detector), which integrates generative models into discriminative detectors. We leverage a point cloud upsampling network with prior knowledge to enhance geometric details of foreground objects, aiding the detector in achieving more accurate prediction. PUDet incorporates two key modules: LDEM (Local Distribution Enhancement Module) for nearby objects, which optimizes point distribution while minimizing computational costs, and DDAM (Distant Density Augmentation Module) for distant objects, which increases point density to better delineate object contours. To validate the optimization of geometric contours, we conducted experiments comparing uniform loss before and after enhancement for both nearby and distant objects, demonstrating the efficacy of LDEM and DDAM. We also display the attention maps on object point clouds, explaining the observed accuracy gains. Experimental results on the KITTI testing set show that our framework improves the baseline CT3D by 1.84 mAP, confirming the effectiveness of PUDet. Code will be available at https://github.com/bellamyhsu/PUDet/tree/main.