f²-ToF: A feature-alignment and frequency-division time-of-flight data denoise network

Infrared wave time-of-flight (ToF) imaging is a important method to sense the 3D information of scene for Internet of Things (IoT) and artificial intelligent (AI). Driven by heavy demands from industry and users, ToF imaging has received significant research attention in recent year, but the artifacts of depth image still remain and need to be improved. Removing multiple artifacts of ToF data is usually treated as a multi-stage stitching problem for deep learning methods. However, the multi-stage cascade and cross-domain refinement architecture could increase the difficulty of model fitting and hurt the effect of noise reduction. In this paper, we classify the artifacts of ToF data as temporal-related or modulation frequency-related noise and propose a ToF denoising convolutional neural network (f²-ToF) to reduce multiple artifacts simultaneously. Specifically, a frequency-division structure is designed to reduce the influence of frequency-related noise in different modulation frequencies. For efficient correcting misalignment data and ensuring a one-stage end-to-end training, the feature-wise alignment module is proposed. In experiments, every proposed module effectively performed its designed task, and the whole framework achieved strong performance in ToF image refinement.