Embedded sign language interpreter system for deaf and dumb people

Abstract

Deaf and dumb conversation is hinged on sign language which renders words through hand and finger symbols. Main objective of the project is building a device that assist deaf and dumb people to convey their messages to normal people. The system consists of two main parts. The first part is hand sign recognition and the second one is speech recognition. This ensures a bi directional communication avoiding the assistance of an interpreter. A webcam for image recognition, a microcontroller unit, and a display device associated with the controller and a Bluetooth module constitutes the hand sign recognition part. In the speech recognition part, digitized signal after processing undergoes statistical modeling and matching for recognition. Speech recognition part is integrated on an easy to use android application. Since the system is deployed in both mobile and embedded platforms, a high performance and power efficient ARM Cortex A8 Processor board is used. For image classification Haar classifier is used while 1-D HMM is used for Speech conversion. Sign language recognition has established its importance in many areas such as Human Computer Interactions (HCIs), robotic controls, home automation etc.