Shortest path with backtracking based automatic layer segmentation in pathological retinal optical coherence tomography images

Optical coherence tomography (OCT) is a high-resolution and non-invasive imaging modality that has become one of the most prevalent techniques for ophthalmic diagnosis. Retinal layer segmentation is crucial for doctors to diagnose and analysis retinal diseases. Manual segmentation is often a time-consuming and subjective process. A number of semi-automatic and automatic methods have been proposed for layer segmentation on retinal OCT images, but very few of them are applicable for retinal pathological OCT images. In this work, we propose a new automatic method for segmenting ILM (Inner Limiting Membrane) and OS-RPE (Outer Segment- Retinal Pigment Epithelium) interfaces on pathological OCT images affected by macular hole disease. The proposed method follows shortest path framework, while is enhanced with backtracking and direction consistency. Backtracking can deal with the shortcut problem which occur when classical shortest path algorithm is used to segment deformed linear structure. “Consistency loss” is one kind of soft constraint we defined to limit the propagate direction of modified shortest path algorithm. Besides, the image information after Gabor transform can reflect the layer location to an extent. So, it is considered as one new weight in the weight calculation of the method. Another contribution is that the proposed layer segmentation method is suitable for both normal and pathological retinal OCT images. To quantitate the performance of the proposed method, we did comparative experiments with three state-of-the-art segmentation methods. The experimental result shows that proposed method can achieve better result than other methods and can deal with pathological retinal OCT images.