Real-time stereo facial feature tracking: mimicking human mouth movement on a humanoid robot head

Abstract

In this paper, we describe a novel real-time stereo facial feature tracking algorithm in presence of geometry and illumination changes. The algorithm is applied to real-time human-humanoid interactions. First, facial feature tracking algorithm locates the facial landmarks such as eyes with the left camera. Then, matching the between the eye pattern on stereo images, the horopter of the head location is estimated for fast stereo disparity processing. Thus, as a result, knowing the disparity and calibration parameters of stereo cameras, not only make it possible to track features in 3D, but also makes tracking more robust. We demonstrate the developed stereo tracking system through a human to humanoid robot mouth mimicking task.