NeRFE: Free View Synthesis for Event Data

Full view perception of surroundings and free view synthesis are of great importance for navigation. Although multiple conventional cameras can provide multi-view stereos of the external environment, limited sensitivity and dynamic range prohibits their applications in scenarios with extreme lighting conditions, such as at night time or in a tunnel. As a bio-inspired device, event camera intrinsically enjoys advantage of ultra-fast response speed, low latency and high dynamic range, but the free view synthesis is nontrivial for the sparse and noisy events. To address this issue, we present a framework for synthesizing novel views freely from event sequences. Specifically, we introduce a deep network representing the neural radiance field of the scene’s event signals for 3D structure encoding. To leverage the sparsity of event data, we introduce an edge based loss function into the optimization process. The learned deep neural network can render novel views which provide additional structures and details to the raw data in original views. We envision that these newly retrieved information can be exploited for further downstream tasks like object detection, tracking and mapping.