Cross-Modal Semidense 6-DOF Tracking of an Event Camera in Challenging Conditions

Vision-based localization is a cost-effective and, thus, attractive solution for many intelligent mobile platforms. However, its accuracy and especially robustness still suffer from low illumination conditions, illumination changes, and aggressive motion. Event-based cameras are bio-inspired visual sensors that perform well in high dynamic range conditions and have high-temporal resolution, and thus, provide an interesting alternative in such challenging scenarios. While purely event-based solutions currently do not yet produce satisfying mapping results, the present work demonstrates the feasibility of purely event-based tracking if an alternative sensor is permitted for mapping. The method relies on geometric 3-D-2-D registration of semidense maps and events, and achieves highly reliable and accurate cross-modal tracking results. Practically relevant scenarios are given by depth camera-supported tracking or map-based localization with a semidense map prior created by a regular image-based visual SLAM or structure-from-motion system. Conventional edge-based 3-D-2-D alignment is extended by a novel polarity-aware registration that makes use of signed time-surface maps obtained from event streams. We, furthermore, introduce a novel culling strategy for occluded points. Both modifications increase the speed of the tracker and its robustness against occlusions or large view-point variations. The approach is validated on many real datasets covering the abovementioned challenging conditions, and compared against similar solutions realized with regular cameras.