How Challenging is a Challenge? CEMS: a Challenge Evaluation Module for SLAM Visual Perception

Xuhui Zhao, Zhi Gao*, Hao Li, Hong Ji, Hong Yang, Chenyang Li, Hao Fang, Ben M. Chen

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)

Abstract

Despite promising SLAM research in both vision and robotics communities, which fundamentally sustains the autonomy of intelligent unmanned systems, visual challenges still threaten its robust operation severely. Existing SLAM methods usually focus on specific challenges and solve the problem with sophisticated enhancement or multi-modal fusion. However, they are basically limited to particular scenes with a non-quantitative understanding and awareness of challenges, resulting in a significant performance decline with poor generalization and(or) redundant computation with inflexible mechanisms. To push the frontier of visual SLAM, we propose a fully computational reliable evaluation module called CEMS (Challenge Evaluation Module for SLAM) for general visual perception based on a clear definition and systematic analysis. It decomposes various challenges into several common aspects and evaluates degradation with corresponding indicators. Extensive experiments demonstrate our feasibility and outperformance. The proposed module has a high consistency of 88.298% compared with annotation ground truth, and a strong correlation of 0.879 compared with SLAM tracking performance. Moreover, we show the prototype SLAM based on CEMS with better performance and the first comprehensive CET (Challenge Evaluation Table) for common SLAM datasets (EuRoC, KITTI, etc.) with objective and fair evaluations of various challenges. We make it available online to benefit the community on our website.

Original languageEnglish
Article number42
JournalJournal of Intelligent and Robotic Systems: Theory and Applications
Volume110
Issue number1
DOIs
Publication statusPublished - Mar 2024
Externally publishedYes

Keywords

  • Challenge evaluation
  • Resilient SLAM
  • Robotics
  • Visual degradation

Fingerprint

Dive into the research topics of 'How Challenging is a Challenge? CEMS: a Challenge Evaluation Module for SLAM Visual Perception'. Together they form a unique fingerprint.

Cite this