Object classification with metric and semantic inference

Abstract

This paper addresses the problem of point cloud segmentation and object (simple and complex) recognition by a mobile robot in realistic indoor environments. In comparison to classical algorithms, the classification method presented by the authors takes into account contextual information of the objects and their mutual spatial relations. In many situations it is impossible to unambiguously attach an observed object segment to a specific class considering only its features. We propose a holistic and developable inference method based on a generalized directional Markov random field that makes use of 3D surface features, intrinsic object parts relations, metric data and semantic relations with other observed objects. Our approach consists of the following stages: feature extraction, segmentation, hypotheses formulation and spatial inference. The presented system allows to easily add more features and semantic relations (or even completely substitute them). Our method has been implemented and tested with real indoor scenes, showing that applying the described inference algorithm significantly improves the results of object classification compared to the feature-only approach.