Topology-aware multi-information fusion for object recognition

Multi-source information fusion plays a crucial role in enhancing object recognition performance. However, in real-world applications such as autonomous driving and industrial inspection, occlusion and data inconsistencies caused by complex environments can undermine the reliability of extracted features. In this paper, we propose a Topology-Aware Multi-Information Fusion (TMF) model, designed to improve the robustness and generalizability of feature extraction in multi-sensor data. For the first time, our model simultaneously integrates topological architectures into both feature extraction and propagation within a multi-source information framework. The proposed model introduces two key modules. The Enhancing Feature Module (EFM) refines local geometric structures in a topology-preserving manner. The Attention Topology Module (ATM) applies topology-aware attention during feature propagation to dynamically recalibrate feature importance, thereby improving the cross-modal fusion process. In addition, 2D RGB features extracted by a lightweight convolutional encoder are concatenated with 3D point-cloud features, providing a clear and effective fusion strategy. Through a structured fusion framework, our method effectively integrates 3D point cloud features with 2D image-based convolutional descriptors, maximizing the complementary advantages of different sensor modalities. Extensive experiments conducted on the S3DIS and Semantic3D datasets validate the effectiveness of our model. Compared to the typical object recognition model, PointNet, our proposed method achieves a 15.3% and 17.1% improvement in mIoU, respectively. Additionally, we further validate the model using self-collected real-world data, demonstrating its applicability across different data distributions and its potential for real-world multi-modal object recognition.