Multi-modal controls of a smart robot

Abstract

Multi-modal controls of robots can be of immense help to the human beings, because of the diverse modes of communication they provide to humans for control of the robots. Ease of operation and user convenience can be other advantages. In this paper, we develop a prototype smart robot whose movements are controlled by voice-commands and gesture-commands. The voice-command controls involve an Android OS based smart phone. The gesture-commands can be either given by hand using an accelerometer, or as the tilt-gestures using the gravity sensor of a smart phone. Robot's body movements are controlled using the voice-commands, and its arms and claws are controlled by the gesture-commands. A comparison is also made between the accelerometer and gravity sensor based gesture-commands, in terms of efficiency and accuracy of the movements of the prototype robot. Human voice-commands are processed in real-time using an online cloud server and are communicated to the smart robot over a Bluetooth network. In a similar way, the gesture-commands generated using either gravity sensor of the smart phone or the accelerometer, are transmitted to the robot over Bluetooth network. Various experiments are conducted on the prototype robot to measure different parameters of its movements, and to evaluate its performance in terms of accuracy and efficiency. Possible applications of the smart robot, its current limitations, and future improvements are also discussed.