TY - GEN
T1 - CLFusion:3D Semantic Segmentation Based on Camera and Lidar Fusion
AU - Wang, Tianyue
AU - Song, Rujun
AU - Xiao, Zhuoling
AU - Yan, Bo
AU - Qin, Haojie
AU - He, Di
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - In the field of autonomous driving, semantic segmentation is crucial for scene understanding. Currently, there are two main methods: camera-based and Lidar-based approaches. To address the issues of Lidar segmentation lacking texture features and image segmentation lacking distance information, this paper proposes a fusion of camera and Lidar to achieve 3D semantic segmentation. The method utilizes a dual-stream encoder-decoder network to process camera images and Lidar point cloud and incorporates a specially designed attention mechanism module for feature fusion. To avoid expensive manual annotation of 3D point clouds, the study also introduces a cross-dataset and cross-modal self-supervised training approach. Experimental results show a 2.4% improvement compared to the Lidar-only mode baseline results on the SemanticKITTI dataset and a 6% improvement on the nuScenes dataset.
AB - In the field of autonomous driving, semantic segmentation is crucial for scene understanding. Currently, there are two main methods: camera-based and Lidar-based approaches. To address the issues of Lidar segmentation lacking texture features and image segmentation lacking distance information, this paper proposes a fusion of camera and Lidar to achieve 3D semantic segmentation. The method utilizes a dual-stream encoder-decoder network to process camera images and Lidar point cloud and incorporates a specially designed attention mechanism module for feature fusion. To avoid expensive manual annotation of 3D point clouds, the study also introduces a cross-dataset and cross-modal self-supervised training approach. Experimental results show a 2.4% improvement compared to the Lidar-only mode baseline results on the SemanticKITTI dataset and a 6% improvement on the nuScenes dataset.
KW - 3D semantic segmentation
KW - multi-modal fusion
KW - self-supervised training
UR - http://www.scopus.com/inward/record.url?scp=85198515270&partnerID=8YFLogxK
U2 - 10.1109/ISCAS58744.2024.10558356
DO - 10.1109/ISCAS58744.2024.10558356
M3 - Conference contribution
AN - SCOPUS:85198515270
T3 - Proceedings - IEEE International Symposium on Circuits and Systems
BT - ISCAS 2024 - IEEE International Symposium on Circuits and Systems
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 IEEE International Symposium on Circuits and Systems, ISCAS 2024
Y2 - 19 May 2024 through 22 May 2024
ER -