PKRT-Net: Prior knowledge-based relation transformer network for optic cup and disc segmentation

Glaucoma causes irreversible vision loss, and early detection of glaucoma is essential to protect the vision of patients. The optic cup (OC) and optic disc (OD) are two critical anatomical structures for glaucoma diagnosis. Methods based on convolutional neural networks (CNNs) have been proposed to extract OC and OD, in which OC extraction is very challenging. However, the clinical prior knowledge is not fully utilized in existing CNN methods, which limits the performance of extracting OC and OD. Besides, CNN methods cannot learn long-range semantic information interaction well due to the intrinsic locality of convolution operations. In this paper, we propose a Prior Knowledge-based Relation Transformer Network (PKRT-Net), which employs the clinical prior knowledge to assist OC segmentation and model efficient long-range relation of spatial features by the transformer. PKRT-Net consists of a dual-branch module, a relation transformer fusion module, and a decoder with weighted fusion. Dual-branch module decouples the fundus image into the vessel feature space and general local feature space; the relation transformer fusion module fuses the clinical prior information with local features to obtain more representative features; the weighted fusion module fuses the multi-scale side-outputs from the decoder with the representation of relation transformer module to improve the segmentation performance. We evaluate our proposed PKRT-Net on three public available OC and OD segmentation datasets (i.e., Drishti-GS, RIM-ONE(r3), and REFUGE). The experimental results demonstrate that our proposed PKRT-Net framework achieves state-of-the-art OC and OD segmentation results on these three public datasets.