Instance-Aware 3D Gaussian Splatting via 2D Segmentation Guidance

3D instance-level data are essential for applications such as virtual reality and game design. As an efficient and high-fidelity 3D scene reconstruction method, 3D Gaussian Splatting (3DGS) provides a promising way to obtain 3D objects from real-world scenes. However, 3DGS do not aware instance information. Existing approaches to segment 3D instance from 3DGS scenes suffer from feature ambiguity and spatial alignment, resulting in artifacts adherent to instance causing inaccurate boundaries. In this paper, we propose a novel effective method to obtain clean 3D instances from real-world scenes. In this method, 2D instance images and masks from 2D segmentation are taken to guide the reconstruction of 3D instances in the training framework of 3DGS, and a multi-view refinement strategy is proposed to refine Gaussians trained by 3DGS according to 2D instance masks. By these means, 3D instances with clear boundaries are successfully extracted from scenes. Experimental results on diverse datasets show that our method significantly improves 3D instance segmentation. Compared with existing methods, it can obtain 3D objects with higher visual quality and fidelity without additional overhead of scene training.