Voice activated command and control with Java-enabled speech recognition over Wifi

Abstract

In this paper we present recent research in the development of a voice enabled command and control application, using Java, in which a remote robotic device, with a JVM, is controlled over a Wifi/WLAN/TCPIP network from a mobile handheld PC (iPaq) or a PC workstation. The research considers three different scenario configurations. A recognition grammar for command and control of the robot has been created and implemented in Java, in part in the recognition engine and in part on the robot. The robot is a proxy for one of a number of JVM enabled intelligent devices that can be controlled in this manner. The physical topology involves Java at each node endpoint, that is, at the handheld PC (iPaq), the PC workstation, the Linux server and onboard the robot (including its Java based Lejos OS). Network communications is primarily WLAN with an element of IR and the possibility of BlueTooth (subject to the resolution of certain important Java and Bluetooth issues). For placement of a speech recognition engine on Linux we are initially using Sphinx through the Java Native Interface but plan to develop our own speech recognition engine, in Java, for deployment on Linux. Traditionally speech processing in software has resided in the domain of C software development. Recent advances in Java with the Java Speech API, along with the Java Media Framework, have facilitated the consideration of Java for speech and language engineering activities. Our hypothesis is that Java is now a suitable platform for speech and language engineering in software. The evidence provided by our initial results, for Java speech recognition, achieved using the Cloudgarden implementation of JSAPI, validates this hypothesis. We outline our future plans to move our framework more fully into the ubiquitous computing arena and extend the reach of Java and speech processing in support of command and control a structured way, including, for example, camera feedback to base.