Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation

Long Shao, Shuo Yang, Tianyu Fu*, Yucong Lin*, Haixiao Geng, Danni Ai, Jingfan Fan, Hong Song, Tao Zhang, Jian Yang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

21 Citations (Scopus)

Abstract

Background: Marker-based augmented reality (AR) calibration methods for surgical navigation often require a second computed tomography scan of the patient, and their clinical application is limited due to high manufacturing costs and low accuracy. Methods: This work introduces a novel type of AR calibration framework that combines a Microsoft HoloLens device with a single camera registration module for surgical navigation. A camera is used to gather multi-view images of a patient for reconstruction in this framework. A shape feature matching-based search method is proposed to adjust the size of the reconstructed model. The double clustering-based 3D point cloud segmentation method and 3D line segment detection method are also proposed to extract the corner points of the image marker. The corner points are the registration data of the image marker. A feature triangulation iteration-based registration method is proposed to quickly and accurately calibrate the pose relationship between the image marker and the patient in the virtual and real space. The patient model after registration is wirelessly transmitted to the HoloLens device to display the AR scene. Results: The proposed approach was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with six advanced AR calibration methods. The proposed method obtained average fusion errors of 0.70 ± 0.16 and 0.91 ± 0.13 mm in phantom and volunteer experiments, respectively. The fusion accuracy of the proposed method is the highest among all comparison methods. A volunteer liver puncture clinical simulation experiment was also conducted to show the clinical feasibility. Conclusions: Our experiments proved the effectiveness of the proposed AR calibration method, and revealed a considerable potential for improving surgical performance.

Original languageEnglish
Article number105826
JournalComputers in Biology and Medicine
Volume148
DOIs
Publication statusPublished - Sept 2022

Keywords

  • Augmented reality
  • Feature triangulation iteration registration
  • Shape feature matching
  • Surgical navigation

Fingerprint

Dive into the research topics of 'Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation'. Together they form a unique fingerprint.

Cite this

Shao, L., Yang, S., Fu, T., Lin, Y., Geng, H., Ai, D., Fan, J., Song, H., Zhang, T., & Yang, J. (2022). Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation. Computers in Biology and Medicine, 148, Article 105826. https://doi.org/10.1016/j.compbiomed.2022.105826