NeRFE: Free View Synthesis for Event Data

Bo Zhang, Yuqi Han, Jinli Suo*, Qionghai Dai

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Full view perception of surroundings and free view synthesis are of great importance for navigation. Although multiple conventional cameras can provide multi-view stereos of the external environment, limited sensitivity and dynamic range prohibits their applications in scenarios with extreme lighting conditions, such as at night time or in a tunnel. As a bio-inspired device, event camera intrinsically enjoys advantage of ultra-fast response speed, low latency and high dynamic range, but the free view synthesis is nontrivial for the sparse and noisy events. To address this issue, we present a framework for synthesizing novel views freely from event sequences. Specifically, we introduce a deep network representing the neural radiance field of the scene’s event signals for 3D structure encoding. To leverage the sparsity of event data, we introduce an edge based loss function into the optimization process. The learned deep neural network can render novel views which provide additional structures and details to the raw data in original views. We envision that these newly retrieved information can be exploited for further downstream tasks like object detection, tracking and mapping.

Original languageEnglish
Title of host publicationAdvances in Guidance, Navigation and Control - Proceedings of 2022 International Conference on Guidance, Navigation and Control
EditorsLiang Yan, Haibin Duan, Yimin Deng, Liang Yan
PublisherSpringer Science and Business Media Deutschland GmbH
Pages6776-6784
Number of pages9
ISBN (Print)9789811966125
DOIs
Publication statusPublished - 2023
Externally publishedYes
EventInternational Conference on Guidance, Navigation and Control, ICGNC 2022 - Harbin, China
Duration: 5 Aug 20227 Aug 2022

Publication series

NameLecture Notes in Electrical Engineering
Volume845 LNEE
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119

Conference

ConferenceInternational Conference on Guidance, Navigation and Control, ICGNC 2022
Country/TerritoryChina
CityHarbin
Period5/08/227/08/22

Keywords

  • Event camera
  • Navigation
  • Neural radiance field

Fingerprint

Dive into the research topics of 'NeRFE: Free View Synthesis for Event Data'. Together they form a unique fingerprint.

Cite this