An Adaptive, Affordable, Humanlike Arm Hand System for Deaf and DeafBlind Communication with the American Sign Language

Abstract

To communicate, the ~ 1.5 million Americans living with deafblindess use tactile American Sign Language (t-ASL). To provide Deafßilind (DB) individuals with a means of using their primary communication language without the use of an interpreter, we developed an assistive technology that promotes their autonomy. The TATUM (Tactile ASL Translational User Mechanism) anthropomorphic arm hand system leverages previous developments of a fingerspelling hand to sign more complex ASL words and phrases. The TATUM hand-wrist system is attached onto a 4 DOF robot arm and a human motion recognition and human to robot gesture transfer framework is used for signing recognition and replication. In particular, signing trajectories based on vision-based motion capture data from a sign demonstrator were used to control the robot's actuators. The performance of the system was evaluated through tactile based sign recognition performed by a blinded user and for its accuracy with novice, sighted users.