Tracking benchmark and evaluation for manipulation tasks

Abstract

In this paper we present a public dataset to evaluate trackers used for human and robot manipulation tasks. For these tasks both high DOF motion and high accuracy is needed. We describe in detail, both the process of recording the sequences and how ground truth data was generated for the videos. The videos are tagged with challenges that a tracker would face while tracking the object. As an initial example, we evaluate the performance of six published trackers [5], [11], [12], [13], [15], [6] and analyse their result. We describe a new evaluation metric to test sensitivity of trackers to speed. A total of 100 annotated and tagged sequences are reported. All the videos, ground truth data, original implementation of trackers and evaluation scripts are made publicly available on the website so others can extend the results on their trackers and evaluation.