On the Development of a Voice and Gesture Based HMI for the Control of a Mobile Robot

Abstract

This paper presents the development of a voice and gesture based interface (HMI – Human - Machine Interface) for the human control of a mobile robot. The application utilizes the Microsoft Kinect sensor and its Visual Studio C# libraries for voice and gestures recognition. Once, the specific voice commands or gestures are recognized, the controller (a PC or Laptop) sends specific commands to a microcontroller (ATmega2560 on an Arduino Mega board). The microcontroller is mounted on the mobile robot and controls the robot’s motors. Also there are presented some limitations of the Kinect sensor to be taken into consideration when using it.