Robust hand tracking based on online learning and multi-cue flocks of features

Abstract

Robust hand tracking is in increasing demand from areas such as natural Human Robot/Computer interaction (HRI/HCI) and surveillance systems, while it is still a great challenge due to human hand's drastic appearance change. In recent years, online learning techniques have shown great potential in learning appearance of objects and tackling occlusion. This paper extends an online learning framework called Tracking-Learning-Detection (TLD) to track human hand. The main extensions are: 1) original tracker is replaced by a hybrid multi-cue base tracker combining Median-Flow tracker and Flock-of-Features (FoF) tracker, 2) skin color cue is integrated into cascade detector and PN online learning for more efficiency. Extensive experiments show that the new framework works with more robustness compared with state-of-the-art hand trackers and also original TLD.