Cross-Modal LiDAR-Visual-Inertial Localization in Prebuilt LiDAR Point Cloud Map Through Direct Projection

Jianghao Leng, Chao Sun*, Bo Wang, Yungang Lan, Zhishuai Huang, Qinyan Zhou, Jiahao Liu, Jiajun Li

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

This article proposes the light detection and ranging (LiDAR) map LiDAR-visual-inertial localization (LM-LVIL) system, integrating LiDAR, visual, and inertial sensors within a preconstructed LiDAR map. The key innovation is cross-modal fusion between camera images and point cloud maps via direct projection. Unlike traditional methods aligning 3-D points triangulated from the camera with the point cloud, our system projects map points into the camera frame to minimize photometric error, similar to direct methods in visual simultaneous localization and mapping (SLAM). The system uses an iterated error state Kalman filter (IESKF) framework, incorporating inertial measurement unit (IMU) integration, visual measurements, and LiDAR measurement for precise pose estimation. LiDAR measurement aligns points by ensuring scan points lie on a local plane in the map. Visual measurements include photometric measurement, projection measurement, 3-D point alignment, and 2-D–3-D line measurement. The experimental results show that LiDAR map visual-inertial localization (LM-VIL) achieves competitive accuracy indoors compared to the state-of-the-art methods. The LM-LVIL system improves accuracy in both indoor and outdoor scenarios compared to the LiDAR map LiDAR-inertial localization (LM-LIL) system and the LiDAR localization baseline method. Our localization system balances accuracy and real-time performance, demonstrating application potential. (Figure presented).

Original languageEnglish
Pages (from-to)33022-33035
Number of pages14
JournalIEEE Sensors Journal
Volume24
Issue number20
DOIs
Publication statusPublished - 2024

Keywords

  • Cross-modal
  • direct method
  • map-based localization
  • multisensor fusion
  • photometric error

Fingerprint

Dive into the research topics of 'Cross-Modal LiDAR-Visual-Inertial Localization in Prebuilt LiDAR Point Cloud Map Through Direct Projection'. Together they form a unique fingerprint.

Cite this