Deep learning-enhanced Colmap for 3D reconstruction and segmentation of facial port-wine stains for comprehensive evaluation

Background: Accurate three-dimensional (3D) reconstruction and segmentation of port-wine stains (PWS) is crucial for effective treatment planning and objective outcome evaluation. Yet, these tasks are challenged by the lesions’ diverse shapes, color heterogeneity, and indistinct boundaries. This study aims to develop an integrated approach for 3D reconstruction and segmentation of PWS lesions to address these challenges. Methods: We propose a novel method that combines deep learning with the Colmap 3D reconstruction algorithm. First, we established a standardized image acquisition system with color calibration to capture high-quality, color-accurate two-dimensional (2D) images, thereby ensuring a robust basis for generating precise 3D PWS point clouds. Second, the 2D images collected from 17 patients were reconstructed into 3D point clouds and converted into closed, easily computable mesh models. Third, the multi-color space adaptive fusion network was employed for 2D lesion segmentation. The 3D lesion morphology was reconstructed by utilizing prior information from their 3D point clouds, and the 3D lesion surface area was subsequently calculated. Results: The proposed 3D reconstruction method achieved an average root mean square error (RMSE) of 0.9611 mm when registered against a structured-light scanning reference. Additionally, the contrastive language-image pre-training (CLIP) similarity score between the 3D mesh model and the corresponding 2D image exceeded 0.92, collectively validating the reconstruction accuracy. The average relative error between the computed 3D lesion surface area and the ground-truth area was 4.59%, outperforming conventional 2D measurement approaches and supporting more reliable quantitative assessment. Conclusions: High-quality 3D PWS lesions with accurate boundaries and superior color fidelity were successfully obtained. Our method integrates deep learning with Colmap to facilitate precise 3D reconstruction and segmentation of PWS lesions, offering a promising tool for PWS assessment and treatment planning.