Task-driven latent active correction for physics-inspired input method in near-field mixed reality applications

Calibration accuracy is one of the most important factors to affect the user experience in mixed reality applications. For a typical mixed reality system built with the optical see-through head-mounted display, a key problem is how to guarantee the accuracy of hand–eye coordination by decreasing the instability of the eye and the head-mounted display in long-term use. In this paper, we propose a real-time latent active correction algorithm to decrease hand–eye calibration errors accumulated over time. Experimental results show that we can guarantee an effective calibration result and improve the user experience with the proposed latent active correction algorithm. Based on the proposed system, experiments about virtual buttons are also designed, and the interactive performance regarding different scales of virtual buttons is presented. Finally, a direct physics-inspired input method is constructed, which shares a similar performance with the gesture-based input method but provides a lower learning cost due to its naturalness.