6D pose estimation and 3D object reconstruction from 2D shape for robotic grasping of objects

Abstract

We propose a method for 3D object reconstruction and 6D pose estimation from 2D images that uses knowledge about object shape as the primary key. In the proposed pipeline, recognition and labeling of objects in 2D images deliver 2D segment silhouettes that are compared with the 2D silhouettes of projections obtained from various views of a 3D model representing the recognized object class. Transformation parameters are computed directly from the 2D images, making the approach feasible. Furthermore, 3D transformations and projective geometry are employed to arrive at a full 3D reconstruction of the object in camera space using a calibrated setup. The method is quantitatively evaluated using synthetic data and tested with real data. In robot experiments, successful grasping of objects demonstrates its usability in real-world environments. The method is applicable to scenarios where 3D object models, e.g., CAD-models or point clouds, are available and precise pixel-wise segmentation maps of 2D images can be obtained. Different from other methods, the method does not use 3D depth for training, widening the domain of application.