A deep learning based approach for Arabic Sign language alphabet recognition using electromyographic signals

Abstract

Unlike spoken languages, Sign languages are mainly based on gestures. This visual aspect makes their recognition a challenge for researchers. Despite the great advances in language processing, sign language recognition still requires a lot of work to achieve acceptable accuracy. On this work we aim to contribute on the improvement of the Arabic sign language recognition. In fact, we aim to exploit the electromyographic EMG signal produced by our muscles while making a gesture to recognize the gesture. Electromyographic signals are rich in information that can be used to calculate muscle activation. They present a potential technology for gesture recognition and classification. These signals are widely used to manufacture human-computer interaction (HMI) devices to control a prosthesis, a robot, or a computer to perform certain predefined tasks. The main objective of our research is to propose a new approach based on deep learning to recognize the Arabic sign language alphabet. We have proposed a novel convolutional neural network that uses LSTM to process feature dependencies. This model is applied to recognize 7 alphabetic characters of Arabic Sign Language. Compared to existing approaches, our approach shows a high accuracy of 97.5^%.