Deep learning-based multimodal CT/MRI image fusion and segmentation strategies for surgical planning of oral and maxillofacial tumors: A pilot study

Purpose: This pilot study aims to evaluate the feasibility and accuracy of deep learning-based multimodal computed tomography/magnetic resonance imaging (CT/MRI) fusion and segmentation strategies for the surgical planning of oral and maxillofacial tumors. Materials and methods: This study enrolled 30 oral and maxillofacial tumor patients visiting our department between 2016 and 2022. All patients underwent enhanced CT and MRI scanning of the oral and maxillofacial region. Furthermore, three fusion models (Elastix, ANTs, and NiftyReg) and three segmentation models (nnU-Net, 3D UX-Net, and U-Net) were combined to generate nine hybrid deep learning models that were trained. The performance of each model was evaluated via the Fusion Index (FI), Dice similarity coefficient (Dice), 95th-percentile Hausdorff distance (HD95), mean surface distance (MSD), precision, and recall analysis. Results: All three image fusion models (Elastix, ANTs, and NiftyReg) demonstrated satisfactory accuracy, with Elastix exhibiting the best performance. Among the tested segmentation models, the highest degree of accuracy for segmenting the maxilla and mandible was achieved by combining NiftyReg and nnU-Net. Furthermore, the highest overall accuracy of the nine hybrid models was observed with the Elastix and nnU-Net combination, which yielded a Dice coefficient of 0.89 for tumor segmentation. Conclusion: In this study, deep learning models capable of automatic multimodal CT/MRI image fusion and segmentation of oral and maxillofacial tumors were successfully trained with a high degree of accuracy. The results demonstrated the feasibility of using deep learning-based image fusion and segmentation to establish a basis for virtual surgical planning.