GCVNet: Geometry Constrained Voting Network to Estimate 3D Pose for Fine-Grained Object Categories

As a fundamental AI problem, monocular 3D pose estimation has received much attention. This paper addresses the challenge of estimating full perspective model parameters, including object pose and camera intrinsics, from a single 2D image of fine-grained object categories. To tackle this highly ill-posed problem, we propose a Geometry Constrained Voting Network (GCVNet). It is a unified end-to-end network consisting of four synergic task-specific subnetworks: 1) Fine-grained classification subnetwork, offering fine-grained 3D shape priors. 2) Voting subnetwork, generating 2D measurements. 3) Segmentation subnetwork, providing a foreground mask for voting. 4) PnP subnetwork, estimating the perspective parameters via explicit geometric reasoning, as well as constraining the classification subnetwork to provide proper 3D priors and the voting subnetwork to generate a group of geometric consistent 2D measurements, rather than independent voting for each 2D measurement in the literature. Experiments on challenging datasets demonstrate the superior performance of GCVNet.