A prototype of a P300 based brain-robot interface to enable multi-modal interaction for patients with limited mobility

Abstract

Patients who lost their ability to move and talk are often socially deprived. To assist them, we present a prototype of a humanoid robotic system that aims to extend the social sphere and autonomy of the patients via an EEG based brain-computer interface. The system enables multi-modal and bidirectional communication. It empowers the patient to interact with the robot and command it by using a high-level P300 BCI that interprets the patient's answers to questions asked by the robot. Additionally, the system allows interaction with other people. By forwarding some of the robot's sensations to the patient, the patient's senses and action space are extended and a telepresence of the patient is created. A use-case validation of the communication system yielded BCI offline classification rates of 93.3% and online classification rates of 70-90%. With a communication rate of two high-level command selections per minute, bidirectional communication between an able-bodied test subject and the robotic system was possible.