Inertial and optical sensor fusion to compensate for partial occlusions in surgical tracking systems

To solve the occlusion problem in optical tracking system (OTS) for surgical navigation, this paper proposes a sensor fusion approach and an adaptive display method to handle cases where partial or total occlusion occurs. In the sensor fusion approach, the full 6D pose information provided by the optical tracker is used to estimate the bias of the inertial sensors when all of the markers are visible. When partial occlusion occurs, the optical system can track the position of at least one marker which can be combined with the orientation estimated from the inertial measurements to recover the full 6D pose information. When all the markers are invisible, the position tracking will be realized based on outputs of the Inertial Measurement Unit (IMU) which may generate increasing drifting error. To alert the user when the drifting error is great enough to influence the navigation, the images adaptive to the drifting error are displayed in the field of the user's view. The experiments are performed with an augmented reality HMD which displays the AR images and the hybrid tracking system (HTS) which consists of an OTS and an IMU. Experimental result shows that with proposed sensor fusion approach the 6D pose of the head with respect to the reference frame can be estimated even under partial occlusion conditions. With the help of the proposed adaptive display method, the users can recover the scene of markers when the error is considered to be relatively high.