Robust fingertip extraction with improved skin color segmentation for finger gesture recognition in Human-robot interaction

Abstract

This paper presents an efficient approach to solve the problem of real-time robust hand segmentation and fingertip extraction for finger gesture recognition in Human-robot interaction (HRI). Firstly, we propose an improved cascade filtering based hand region candidate estimation by the combination of YC <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</inf> C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">r</inf> skin color probability image based segmentation with genetic algorithm and post-processing with morphological operation and blob analysis in blurred, low-resolution images. Using the segmented hand candidate regions, we estimate the hand region center and fingertip position from distance transform and geometrical feature of hand. From the hand orientation and hand/palm center, we find the optimal each fingertip position and its orientation. Experimental results show that the proposed algorithm not only rapidly detects the hand regions under various illumination conditions, but also it efficiently extracts the finger information with size and rotation invariance.