Prior-guided local geometric modeling vessel segmentation network for OCT images

Automated vessel segmentation from OCT images is critical for biomarker measurement of vasculature and understanding the progression and treatment of many ocular diseases. Existing vessel segmentation methods mainly focus on pixel-level segmentation, ignoring the inherent structural prior of vessels. Some prior-based segmentation methods have introduced prior knowledge into deep networks to improve the structural representation performance. However, they tend to encode the global prior distribution of the mask, but struggle to model the local geometric structure of microvessels and interact effectively. To alleviate these challenges, we propose a novel prior-guided local geometric modeling framework for OCT vessel segmentation. The proposed framework is able to model microvessel structure prior and achieve effective interaction between image features and mask features. The proposed framework consists of three main components: (1) a sparse codebook that quantizes and stores vessel prior features, increasing the sparsity of discrete vectors in the codebook by introducing a contrast learning loss; (2) a principal component alignment loss that implements feature coarse matching by mapping prior and image features into principal component space, mitigating the overfitting caused by forcibly matching image features with prior features; (3) a new local linear embedding loss that performs local geometric structure alignment of the mask prior with the replaced latent features, to enhance the continuity of vessels in the segmentation result. The proposed framework allows embedding into arbitrary encoder–decoder networks and has been extensively validated on one public and three in-house datasets involving multiple OCT instruments. Experimental results show that our method achieves the state-of-the-art performance compared to dedicated vessel segmentation methods and prior-based methods. In addition, we perform a comprehensive parameter selection and visualization analysis to enhance the interpretability of the proposed framework.