Towards Robust Neuroadaptive HCI: Exploring Modern Machine Learning Methods to Estimate Mental Workload From EEG Signals

Estimating mental workload from brain signals such as Electroencephalography (EEG) has proven very promising in multiple Human-Computer Interaction (HCI) applications, e.g., to design games or educational applications with adaptive difficulty, or to assess how cognitively difficult to use an interface can be. However, current EEG-based workload estimation may not be robust enough for some practical applications. Indeed, the currently obtained workload classification accuracies are relatively low, making the resulting estimations not fully trustable. This paper thus studies promising modern machine learning algorithms, including Riemannian geometry-based methods and Deep Learning, to estimate workload from EEG signals. We study them with both user-specific and user-independent calibration, to go towards calibration-free systems. Our results suggested that a shallow Convolutional Neural Network obtained the best performance in both conditions, outperforming state-of-the-art methods on the used data sets. This suggests that Deep Learning can bring new possibilities in HCI.

[1]  F. Yger,et al.  Riemannian Approaches in Brain-Computer Interfaces: A Review , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Daniel Afergan,et al.  Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State , 2016, CHI.

[3]  Jérémy Frey,et al.  Framework for Electroencephalography-based Evaluation of User Experience , 2016, CHI.

[4]  Robert C. Wolpert,et al.  A Review of the , 1985 .

[5]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing to evaluate information visualization interfaces , 2013, CHI.

[6]  Brian C. Ross Mutual Information between Discrete and Continuous Data Sets , 2014, PloS one.

[7]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[8]  Jérémy Frey,et al.  TOBE: Tangible Out-of-Body Experience , 2015, TEI.

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  Maureen Clerc,et al.  Brain-Computer Interfaces 2: Technology and Applications , 2016 .

[11]  S. Fairclough BCI and Physiological Computing for Computer Games : Differences , Similarities & Intuitive Control , 2009 .

[12]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[13]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[14]  Volker Tresp,et al.  Tensor-Train Recurrent Neural Networks for Video Classification , 2017, ICML.

[15]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[16]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[17]  Daniel S. Margulies,et al.  Prioritizing spatial accuracy in high-resolution fMRI data using multivariate feature weight mapping , 2014, Front. Neurosci..

[18]  Thibault Gateau,et al.  Real-Time State Estimation in a Flight Simulator Using fNIRS , 2015, PloS one.

[19]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[20]  Christian Jutten,et al.  Multiclass Brain–Computer Interface Classification by Riemannian Geometry , 2012, IEEE Transactions on Biomedical Engineering.