A Mixed Classification-Regression Framework for 3D Pose Estimation from\n 2D Images

Abstract

3D pose estimation from a single 2D image is an important and challenging\ntask in computer vision with applications in autonomous driving, robot\nmanipulation and augmented reality. Since 3D pose is a continuous quantity, a\nnatural formulation for this task is to solve a pose regression problem.\nHowever, since pose regression methods return a single estimate of the pose,\nthey have difficulties handling multimodal pose distributions (e.g. in the case\nof symmetric objects). An alternative formulation, which can capture multimodal\npose distributions, is to discretize the pose space into bins and solve a pose\nclassification problem. However, pose classification methods can give large\npose estimation errors depending on the coarseness of the discretization. In\nthis paper, we propose a mixed classification-regression framework that uses a\nclassification network to produce a discrete multimodal pose estimate and a\nregression network to produce a continuous refinement of the discrete estimate.\nThe proposed framework can accommodate different architectures and loss\nfunctions, leading to multiple classification-regression models, some of which\nachieve state-of-the-art performance on the challenging Pascal3D+ dataset.\n