Splatting-Based View Synthesis for Self-supervised Monocular Depth Estimation

Self-supervised method has shown great potential in monocular depth estimation, since it does not need expensive ground-truth depth labels but only uses the photometric error of synthesized images as the supervision signal. However, although many methods have been proposed to improve its performance, the occlusion problem has not been clearly handled. This paper introduces a novel view synthesis module to deal with occluded pixels in the process of image reconstruction. Specifically, we use bilinear splatting to forward warp the source image, and average pixels projected to the same location by the predicted depth. In addition, a valid pixel mask is generated with projection to ignore invalid pixels. The proposed approach clearly handles overlapping pixels and invalid areas of the synthesized image, thus improving the performance of self-supervised learning. We conduct various experiments, and the results show that our model can generate clear and complete depth maps and achieves state-of-the-art performance.