Early childhood education by hand gesture recognition using a smartphone based robot

Abstract

We propose a light and fast hand gesture recognition method using geometric feature for a smartphone based robot and apply it to early childhood mathematics education. The feature of hand gesture is defined by the number of extrema in the plot for the distances between the center point of hand and the outer points of hand from active contour model or snakes. The region of interest (ROI) is continuously updated by Continuously Adaptive Mean Shift Algorithm (CamShift) algorithm, and the snake model is used to make the outer points sequential efficiently. A mathematics learning application for an Android OS smartphone based robot is developed using the hand gesture recognition algorithm. The experiment with Korean children (5-6 years of age) is conducted to evaluate if hand gesture based HRI could promote their mathematics learning. The result suggests that the idea of hand gesture based HRI for early childhood education is feasible and that children can learn mathematics by hand gesture based interaction with a robot.