GRNet: Geometric relation network for 3D object detection from point clouds

Rapid detection of 3D objects in indoor environments is essential for indoor mapping and modeling, robotic perception and localization, and building reconstruction. 3D point clouds acquired by a low-cost RGB-D camera have become one of the most commonly used data sources for 3D indoor mapping. However, due to the sparse surface, empty object center, and various scales of point cloud objects, 3D bounding boxes are challenging to be estimated and located accurately. To address this, geometric shape, topological structure, and object relation are commonly employed to extract box reasoning information. In this paper, we describe the geometric feature among object points as an intra-object feature and the relation feature between different objects as an inter-object feature. Based on these two features, we propose an end-to-end point cloud geometric relation network focusing on 3D object detection, which is termed as geometric relation network (GRNet). GRNet first extracts intra-object and inter-object features for each representative point using our proposed backbone network. Then, a centralization module with a scalable loss function is proposed to centralize each representative object point to its center. Next, proposal points are sampled from these shifted points, following a proposal feature pooling operation. Finally, an object-relation learning module is applied to predict bounding box parameters. Such parameters are the additive sum of prediction results from the relation-based inter-object feature and the aggregated intra-object feature. Our model achieves state-of-the-art 3D detection results with 59.1% mAP@0.25 and 39.1% mAP@0.5 on ScanNetV2 dataset, 58.4% mAP@0.25 and 34.9% mAP@0.5 on SUN RGB-D dataset.