Model-based human pose estimation with spatio-temporal inferencing

Abstract

This thesis presents a computational framework for human pose estimation from depth video sequences. The framework has a potential to achieve interesting applications such as robot motion retargeting, activity recognition, etc, wherever joint motion is an appropriate representation of the human motion. On the one hand, feature points that are informative for pose estimation are tracked with depth image analysis. Human poses are reconstructed from these feature points with kinematic constraints including joint limits and self-collision avoidance. On the other hand, human poses could be estimated based on local optimization using dense correspondences between 3D data and the articulated human model. Both could be unified with temporal motion prediction based on Bayesian information integration. We demonstrate our results for humanoid robot motion learning through a novel collision-free retargeting as well as for an example of the human pose estimation with environmental clutters. We show the computational results on a set of challenging motions where limbs interact with each other.