A Hidden Markov Model Based Sensor Fusion Approach for Recognizing Continuous Human Grasping Sequences

Abstract

The Programming by Demonstration (PbD) technique aims at teaching a robot to accomplish a task by learning from a human demonstration. In a manipulation context, recognizing the demonstrator’s hand gestures, specifically when and how objects are grasped, plays a significant role. Here, a system is presented that uses both hand shape and contact point information obtained from a data glove and tactile sensors to recognize continuous human grasp sequences. The sensor fusion, grasp classification and task segmentation are made by a Hidden Markov Model recognizer that distinguishes 14 grasp types, as presented in Kamakura’s taxonomy. An accuracy of up to 92.2% for a single user system, and 90.9% for a multiple user system could be achieved.