3D scene reconstruction from a limited number of viewpoints

Learning based 3D scene reconstruction methods have achieved impressive performance on reconstructing complex geometry and low-textured regions in indoor scenes. These methods typically rely on dense views with abundant viewpoints which cover the whole scene to constrain the learning process. However, less works pay attention on indoor scene reconstruction from sparse views with a limited number of viewpoints which are readily accessible. In this paper, we propose a 3D scene reconstruction method that is able to reconstruct indoor scenes from sparse views with a limited number of viewpoints. We introduce geometry and data priors to address the challenge of insufficient supervision of 3D representations using only 2D images. Specifically, we introduce geometry priors, i.e. , points, lines, and planes that constrain highly structured information, to guide the learning process of 3D representations and alleviate the noise sensitivity for observed regions. We construct an optimization algorithm that jointly optimizes the points, lines and planes, exploiting the inherent interrelationships between these elements. We further introduce data priors, i.e. , features extracted by pre-trained networks, which provides additional information for learning more complete representations and provide completion ability for unobserved regions. Experiments on the ScanNet and 7-Scenes datasets show that our method outperforms previous methods when only using sparse views with a limited number of viewpoints. Experiments of reconstruction from different sparsity levels images with a limited number of viewpoints demonstrate the superiority of our method.