Dynamic sign language and voice recognition for smart home interactive application

Abstract

This paper presents a system for recognition of dynamic sign language and voice recognition for smart home interactive applications. We use the Bag-of-Features and a local part model approach for bare hand dynamic gesture recognition from video. We use a dense sampling to extract local 3D multiscale whole-part features. We adopted three-dimensional histograms of a gradient orientation (3D HOG) descriptor to represent features. The K-means++ method was applied to cluster the visual words. Dynamic hand gesture classification was conducted by using a Bag-of-features (BOF) and non-linear support vector machine (SVM) methods. As the BOF does not track the order of events we use a multiscale local part model to preserve temporal context. Initial experimental results show a higher level of recognition. A voice recognition system is then used to translate voice commands complementing the hand gesture commands for the human-intuitive control of a personal service android robot for smart home or long-term healthcare environment applications.