Fusion of multimodal visual cues for model-based object tracking

摘要

While many robotic applications rely on visual tracking, conventional single-cue algorithms typically fail outside limited tracking conditions. Fusion of multimodal visual cues with complementary failure modes allows tracking to continue despite losing individual cues. While previous applications have addressed multi-cue 2D featurebased tracking, this paper develops a fusion scheme for 3D model-based tracking using a Kalman filter framework. Our algorithm fuses colour, edge and texture cues predicted from a textured CAD model of the tracked object to recover the 3D pose. The fusion framework allows additional cues to be integrated provided a suitable measurement function exists. We also show how measurements from multiple cameras can be integrated without requiring explicit correspondences between views. Experimental results demonstrate the increased robustness achieved by fusing multiple cues.