Combining complementary edge, keypoint and color features in model-based tracking for highly dynamic scenes

Abstract

This paper focuses on the issue of estimating the complete 3D pose of the camera with respect to a complex object, in a potentially highly dynamic scene, through model-based tracking. We propose to robustly combine complementary geometrical edge and point features with color based features in the minimization process. A Kalman filtering and pose prediction process is also suggested to handle potential large interframe motions. In order to deal with complex 3D models, our method takes advantage of hardware acceleration. Promising results, outperforming classical state-of-art approaches, have been obtained on various real and synthetic image sequences, with a focus on space robotics applications.