Surface and Volume Fusion Rendering for Augmented Reality Based Functional Endoscopic Sinus Surgery

Functional endoscopic sinus surgery (FESS) is widely used in head and neck clinical surgery. The nasal cavity is intraoperatively visualised using an endoscope. However, the correct identification of complex structures and the perception of key target spatial relationship are difficult to perform using 2D endoscopic images. Surgeons need to visualise a 3D structure from endoscopic images and patients' preoperative computed tomography (CT) images. Therefore, this paper presents a fusion rendering method for augmented reality based on endoscopic imaging. Motion consistency was performed to improve the number and accuracy of texture-less endoscopic image matching. The gradient optimisation of volume data was used to enhance the rendering and improve the distance perception of multi-layer information. The surface fusion error of the reconstructed surface and CT extraction reached 0.58mm, 3.86mm, and 4.03mm in the model data, cadaver skull data and clinical data, respectively. Various experimental results proved that our method can provide the accurate surface structure of the nasal cavity and can effectively improve the depth distinction of multiple objects for clinical surgery.