HQ-Net: A heatmap-based query backbone for point cloud understanding

With the advancement of sensor technology, including LiDAR and RGB-D cameras, tasks related to point cloud understanding have gained widespread application. The sampling and learning of feature points play a crucial role in point cloud understanding, representing a fundamental and essential aspect of 3D computer vision. In the realm of point cloud processing, traditional methods such as Farthest Point Sampling (FPS) are frequently employed to select feature points. These points are then used in conjunction with effective feature aggregation operators to facilitate reasoning. However, these traditional sampling methods primarily focus on the Euclidean distance between points, lacking a comprehensive consideration of point cloud features. To address this limitation, we introduce HQ-Net, a novel point cloud understanding model designed to optimize feature point sampling using a Heatmap-based Query Backbone. The Heatmap-based Query Backbone utilizes the Heatmap Module to select category-specific point features, effectively generating a heatmap tailored to the category features and automatically extracting features from the heatmap. Subsequently, the proposed Query Encoder is employed to facilitate the interaction between point features and query features, providing mutual guidance and enhancing the model's ability to capture spatial structure and semantic information within point clouds. Additionally, we propose task-specific heads, such as classification and segmentation heads for different tasks, which can be seamlessly integrated with the backbone to achieve excellent performance. In various point cloud understanding tasks, including shape classification, part segmentation, and semantic segmentation, HQ-Net demonstrates excellent performance.