Unsupervised Deep Plane-Aware Multi-homography Learning for Image Alignment

Due to its feature representation capabilities, deep learning has been applied to homography estimation in the field of image alignment. Most deep homography learning methods focus on estimating a single global homography, and cannot deal with the problem of parallax when the scene contains multiple different planes, and the translation of the camera’s optical center is not negligible. In this paper, we propose an unsupervised multi-homography learning method with a plane-perception trait to mitigate this parallax problem. In our model, the problem of multi-homography learning and plane perception are jointly considered, which can benefit from each other. To make the learning process stable under unsupervised setting, we design a special attention mechanism to bootstrap the collaboration between multi-homography learning and plane perception. We construct a new dataset that is captured in real scenes, having many challenges such as multiple planes, large parallax, etc. Quantitative and qualitative results show that our proposed method can better align images with large parallax and multiple planes.