TY - JOUR
T1 - Cross-Modal LiDAR-Visual-Inertial Localization in Prebuilt LiDAR Point Cloud Map Through Direct Projection
AU - Leng, Jianghao
AU - Sun, Chao
AU - Wang, Bo
AU - Lan, Yungang
AU - Huang, Zhishuai
AU - Zhou, Qinyan
AU - Liu, Jiahao
AU - Li, Jiajun
N1 - Publisher Copyright:
© 2001-2012 IEEE.
PY - 2024
Y1 - 2024
N2 - This article proposes the light detection and ranging (LiDAR) map LiDAR-visual-inertial localization (LM-LVIL) system, integrating LiDAR, visual, and inertial sensors within a preconstructed LiDAR map. The key innovation is cross-modal fusion between camera images and point cloud maps via direct projection. Unlike traditional methods aligning 3-D points triangulated from the camera with the point cloud, our system projects map points into the camera frame to minimize photometric error, similar to direct methods in visual simultaneous localization and mapping (SLAM). The system uses an iterated error state Kalman filter (IESKF) framework, incorporating inertial measurement unit (IMU) integration, visual measurements, and LiDAR measurement for precise pose estimation. LiDAR measurement aligns points by ensuring scan points lie on a local plane in the map. Visual measurements include photometric measurement, projection measurement, 3-D point alignment, and 2-D–3-D line measurement. The experimental results show that LiDAR map visual-inertial localization (LM-VIL) achieves competitive accuracy indoors compared to the state-of-the-art methods. The LM-LVIL system improves accuracy in both indoor and outdoor scenarios compared to the LiDAR map LiDAR-inertial localization (LM-LIL) system and the LiDAR localization baseline method. Our localization system balances accuracy and real-time performance, demonstrating application potential. (Figure presented).
AB - This article proposes the light detection and ranging (LiDAR) map LiDAR-visual-inertial localization (LM-LVIL) system, integrating LiDAR, visual, and inertial sensors within a preconstructed LiDAR map. The key innovation is cross-modal fusion between camera images and point cloud maps via direct projection. Unlike traditional methods aligning 3-D points triangulated from the camera with the point cloud, our system projects map points into the camera frame to minimize photometric error, similar to direct methods in visual simultaneous localization and mapping (SLAM). The system uses an iterated error state Kalman filter (IESKF) framework, incorporating inertial measurement unit (IMU) integration, visual measurements, and LiDAR measurement for precise pose estimation. LiDAR measurement aligns points by ensuring scan points lie on a local plane in the map. Visual measurements include photometric measurement, projection measurement, 3-D point alignment, and 2-D–3-D line measurement. The experimental results show that LiDAR map visual-inertial localization (LM-VIL) achieves competitive accuracy indoors compared to the state-of-the-art methods. The LM-LVIL system improves accuracy in both indoor and outdoor scenarios compared to the LiDAR map LiDAR-inertial localization (LM-LIL) system and the LiDAR localization baseline method. Our localization system balances accuracy and real-time performance, demonstrating application potential. (Figure presented).
KW - Cross-modal
KW - direct method
KW - map-based localization
KW - multisensor fusion
KW - photometric error
UR - http://www.scopus.com/inward/record.url?scp=105001207225&partnerID=8YFLogxK
U2 - 10.1109/JSEN.2024.3415170
DO - 10.1109/JSEN.2024.3415170
M3 - Article
AN - SCOPUS:105001207225
SN - 1530-437X
VL - 24
SP - 33022
EP - 33035
JO - IEEE Sensors Journal
JF - IEEE Sensors Journal
IS - 20
ER -