Automated segmentation of overlapped nuclei using concave point detection and segment grouping

Nuclei assessment and segmentation are essential in many biological research applications, but it is a challenge to segment overlapped nuclei. In this paper, a new automatic method is proposed to segment overlapped nuclei robustly and efficiently. The proposed method mainly contains four steps: contour extraction, concave point detection, contour segment grouping and ellipse fitting. Blurry nuclei splitting and unobvious concave point detection are always difficult problems in nuclei segmentation. Contour extraction algorithm provides a smooth contour result and it is employed to estimate the blurriness degree of the image. The blurry level determines parameters in subsequent steps, which improves the accuracy of blurry nuclei splitting. Different methods to extract obvious and unobvious concave points from candidate points are proposed. In addition, grouping rules are proposed to assign segments divided by concave points into groups. Comparison study is performed and experimental results showed the effectiveness of the proposed method.