TY - JOUR
T1 - BiMAConv
T2 - Bimodal Adaptive Convolution for Multispectral Point Cloud Segmentation
AU - Zhang, Zheng
AU - Xu, Tingfa
AU - Lou, Peng
AU - Lv, Peng
AU - Tian, Tiehong
AU - Li, Jianan
N1 - Publisher Copyright:
© 2004-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Multispectral point cloud segmentation, leveraging both spatial and spectral information to classify individual points, is crucial for applications such as remote sensing, autonomous driving, and urban planning. However, existing methods primarily focus on spatial information and merge it with spectral data without fully considering their differences, limiting the effective use of spectral information. In this letter, we introduce a novel approach, bimodal adaptive convolution (BiMAConv), which fully exploits information from different modalities, based on the divide-and-conquer philosophy. Specifically, BiMAConv leverages the spectral features provided by the spectral information divergence (SID) and the weight information provided by the modal-weight block (MW-Block) module. The SID highlights slight differences in spectral information, providing detailed differential feature information. The MW-Block module utilizes an attention mechanism to combine generated features with the original point cloud, thereby generating weights to maintain learning balance sharply. In addition, we reconstruct a large-scale urban point cloud dataset GRSS_DFC_2018_3D based on dataset GRSS_DFC_2018 to advance the field of multispectral remote sensing point cloud, with a greater number of categories, more precise annotations, and registered multispectral channels. BiMAConv is fundamentally plug-and-play and supports different shared-multilayer perceptron (MLP) methods with almost no architectural changes. Extensive experiments on GRSS_DFC_2018_3D and Toronto-3D benchmarks demonstrate that our method significantly boosts the performance of popular detectors.
AB - Multispectral point cloud segmentation, leveraging both spatial and spectral information to classify individual points, is crucial for applications such as remote sensing, autonomous driving, and urban planning. However, existing methods primarily focus on spatial information and merge it with spectral data without fully considering their differences, limiting the effective use of spectral information. In this letter, we introduce a novel approach, bimodal adaptive convolution (BiMAConv), which fully exploits information from different modalities, based on the divide-and-conquer philosophy. Specifically, BiMAConv leverages the spectral features provided by the spectral information divergence (SID) and the weight information provided by the modal-weight block (MW-Block) module. The SID highlights slight differences in spectral information, providing detailed differential feature information. The MW-Block module utilizes an attention mechanism to combine generated features with the original point cloud, thereby generating weights to maintain learning balance sharply. In addition, we reconstruct a large-scale urban point cloud dataset GRSS_DFC_2018_3D based on dataset GRSS_DFC_2018 to advance the field of multispectral remote sensing point cloud, with a greater number of categories, more precise annotations, and registered multispectral channels. BiMAConv is fundamentally plug-and-play and supports different shared-multilayer perceptron (MLP) methods with almost no architectural changes. Extensive experiments on GRSS_DFC_2018_3D and Toronto-3D benchmarks demonstrate that our method significantly boosts the performance of popular detectors.
KW - Adaptive convolution
KW - dataset processing
KW - multispectral
KW - point cloud semantic segmentation
UR - http://www.scopus.com/inward/record.url?scp=105004052004&partnerID=8YFLogxK
U2 - 10.1109/LGRS.2025.3565739
DO - 10.1109/LGRS.2025.3565739
M3 - Article
AN - SCOPUS:105004052004
SN - 1545-598X
VL - 22
JO - IEEE Geoscience and Remote Sensing Letters
JF - IEEE Geoscience and Remote Sensing Letters
M1 - 5001705
ER -