Deep Learning based Command Pointing direction estimation using a single RGB Camera

Abstract

Gesture based communication with humanoid robots, especially using hand gestures has universal appeal and could make humanoid robots share home space in a social environment if the technology can be developed in a matured fashion. Till date conventional vision based gesture recognition has had poor success rate since hand engineered features failed to communicate the gesture semantics unambiguously, especially when the gesture is dynamic in nature. This research proposes a deep convolutional neural network based technique that is able to estimate the command direction represented by a finger pointing gesture, with a considerably high accuracy. Moreover, the proposed architecture is able to estimate the direction of finger pointing using only a single two-dimensional RGB image of the hand gesture without using any depth information or stereo-vision techniques making the system computationally and physically less expensive which is particularly suitable for the real time applications such as communicating with the humanoid robots in a home environment.