Self-Supervised Monocular Visual Odometry Based on Multi-View Spatio-Temporal Feature Fusion

Learning-based visual odometry (VO) estimates the ego-motion of a camera by leveraging consistent pixel movements between pairs of consecutive image frames. Unlike most existing VOs that only concentrate on a single imaging plane, our method, MultiSTVO, utilizes the physical prior of camera motions to mine and fuse enhanced temporal motion features from multiple views. To simultaneously focus on pixel movements under various views, the Multi-View Spatio-Temporal Feature Enhancement is proposed to fully extract features sensitive to different observation views. In addition, the graph attention network-based Pose Graph Attention Refinement is also designed to achieve refined selection and fusion. This meticulous integration process helps to extract substantial and robust motion information to realize the full utilization of motion features. Experiments on KITTI / Málaga demonstrate the promising performance of MultiSTVO. Compared with state-of-the-art other methods, it achieves improvements of 28.9% and 43.1% in translation and rotation evaluations, respectively.