A smart camera with gesture recognition and SLAM capabilities for mobile robots

Abstract

This paper presents a smart camera for a real-time gesture recognition system. The smart camera has been designed and implemented as a system-on-a-chip (SoC), using reconfigurable computing technology. In this system the gesture images are captured by means of a CMOS digital camera. After some pre-processing steps, those are sent to a fault tolerant module (FTM) for the actual recognition process. The FTM implements a RAM-based neural network, using three knowledge bases. In addition, a majority voting technique is used to improve the confidence level in the recognition step. A number of experiments using a prototype implementation and selected gestures resulted in a rate of 100% true recognition. The system also showed to be robust, and flexible as new gesture patterns can be easily included by means of the on-chip training capabilities. An application for this systems is also presented, consisting of four smart cameras used in simultaneous localization and mapping (SLAM) tasks for robotics.