Localization and Tracking of User-Defined Points on Deformable Objects\n for Robotic Manipulation

Abstract

This paper introduces an efficient procedure to localize user-defined points\non the surface of deformable objects and track their positions in 3D space over\ntime. To cope with a deformable object's infinite number of DOF, we propose a\ndiscretized deformation field, which is estimated during runtime using a\nmulti-step non-linear solver pipeline. The resulting high-dimensional energy\nminimization problem describes the deviation between an offline-defined\nreference model and a pre-processed camera image. An additional regularization\nterm allows for assumptions about the object's hidden areas and increases the\nsolver's numerical stability. Our approach is capable of solving the\nlocalization problem online in a data-parallel manner, making it ideally\nsuitable for the perception of non-rigid objects in industrial manufacturing\nprocesses.\n