A 3D Color Reconstruction Model Based on NeRF

This paper proposes a novel 3D color reconstruction model based on an improved NeRF triplane representation. The model performs joint modulation-based encoding and decoding of a single input photograph and the 3D model to derive a triplane representation that incorporates the color features from the image. This triplane is then queried and decoded using the vertex coordinates of the 3D model, ultimately yielding a complete 3D geometry with reconstructed vertex colors. In contrast to traditional triplane-NeRF methods, which require sampling and decoding numerous points in 3D space, our model generates the triplane by learning the features of the 3D model itself, enabling direct sampling and color acquisition specifically at the mesh vertices. This approach significantly enhances surface color reconstruction accuracy. Experimental results demonstrate that the model achieves high reconstruction fidelity for surface colors within the distribution of the training dataset. Furthermore, the proposed architecture yields superior reconstruction accuracy and robustness compared to baseline 3D reconstruction models employing traditional NeRF methodologies.