Label super resolution for 3D magnetic resonance images using deformable U-net

Robust and accurate segmentation results from high resolution (HR) 3D Magnetic Resonance (MR) images are desirable in many clinical applications. State-of-the-art deep learning methods for image segmentation require external HR atlas image and label pairs for training. However, the availability of such HR labels is limited due to the annotation accuracy and the time required to manually label. In this paper, we propose a 3D label super resolution (LSR) method which does not use an external image or label from a HR atlas data and can reconstruct HR annotation labels only reliant on a LR image and corresponding label pairs. In our method, we present a Deformable U-net, which uses synthetic data with multiple deformation for training and an iterative topology check during testing, to learn a label slice evolving process. This network requires no external HR data because a deformed version of the input label slice acquired from the LR data itself is used for training. The trained Deformable U-net is then applied to through-plane slices to estimate HR label slices. The estimated HR label slices are further combined by label a fusion method to obtain the 3D HR label. Our results show significant improvement compared to competing methods, in both 2D and 3D scenarios with real data.