TRO-VO: Transformers for Feature Refinement and Pose Optimization in Monocular Visual Odometry

Monocular visual odometry based on self-supervised learning enables autonomous vehicles to estimate precise self-motion in challenging environments without stable satellite signals. To address the accumulated error typically associated with the limited receptive field of monocular setups, our proposed method, named transformers for feature refinement and pose optimization in monocular visual odometry (TRO-VO), integrates learning-based pose prediction, motion feature refinement, and pose graph optimization using transformer-based networks. In the first stage, TRO-VO extracts feature-level changes in the Deformable Pose Former (DPF), revealing the relationship between two consecutive images. Subsequently, a Pose Graph Former (PGF) is devised to optimize the predicted motions and refine the feature maps. A graph-based pose consistency loss function is used to enforce geometric consistency in the optimized motions. Furthermore, TRO-VO enforces separate pose predictions from the DPF and PGF modules: coarse-level perception from DPF and fine-level refinement from PGF, thereby enhancing the interpretability of the entire pipeline. Experimental results on KITTI, RobotCar datasets, as well as self-collected outdoor data demonstrate the performance and generalization capabilities of TRO-VO.