Mental Workload Classification From Spatial Representation of FNIRS Recordings Using Convolutional Neural Networks

Abstract

Mental workload classification is a core element of designing adaptive Human-Computer Interfaces and plays an essential role in increasing the safety and operator performance of complex high-precision human-machine systems in fields such as aerospace and robotic surgery. Among noninvasive neuroimaging techniques, functional Near Infrared Spectroscopy (fNIRS) is a promising sensing modality for decoding mental states. While a variety of both classical and more modern classification techniques have been explored for fNIRS data, Convolutional Neural Networks (CNNs) have received only minimal attention. A significant advantage of CNNs compared to other classification methods is that they don't require prior feature selection or computationally demanding preprocessing. In previous studies on using CNN for fNIRS signals, temporal information from the fNIRS time series was emphasized, but valuable spatial information contained in the recordings was neglected. In this work, we propose and evaluate new structures for the image data fed to the CNN. We exploit the spatial information available in the fNIRS data by constructing images that retain spatial structure. Classification results on real datasets show a significant improvement (16% and 8%) compared to existing Support Vector Machine and Deep Neural Network methods.