Structure and motion by sensor fusion for autonomous navigation system

Real-time structure and motion is the most important research direction, which can be applied in vehicle navigation, spacecraft landing, intelligent monitoring system. Existing vision based structure and motion algorithms are too fragile and tend to drift. How the fusion of inertial and vision data can be used to gain robustness is investigated. The fusion is based on Kalman filtering, using an Extended Kalman filter to fuse inertial and vision data, and a bank of Kalman filters to estimate the sparse 3D structure of the real scene. Two frame feature-based motion estimation is used for initial pose estimation. The motion and structure estimation filters work alternately to recover the sensor motion, scene structure and other parameters. The performance of this algorithm has been tested on real image sequences. Experimental results show that additional inertial data not only can be used to improve position accuracy of reconstructed features and motion estimation, but also can enhance the robustness of the algorithm.