WLnet: Towards an Approach for Robust Workload Estimation Based on Shallow Neural Networks

Electroencephalography (EEG) is a non-invasive technology used for the human brain-computer interface. One of its important applications is the evaluation of the mental state of an individual, such as workload estimation. In previous works, common spatial pattern feature extraction methods have been proposed for the EEG-based workload detection. Recently, several novel methods were introduced to detect EEG pattern workloads. However, it is still unknown which one of these methods is the one that offers the best performance for the workload EEG pattern feature detection. In this article, four methods were used to extract workload EEG features: (a) common spatial pattern feature extraction; (b) temporally constrained sparse group spatial pattern feature extraction; (c) EEGnet; and (d) the new proposed shallow convolutional neural network for workload estimation (WLnet). The classification accuracy of these four methods was compared. Experimental results demonstrate that the proposed WLnet achieved the best detection accuracy in both stress and non-stress conditions. We believe that the proposed methods may be relevant to real-life applications of mental workload estimation.

[1]  G. Pfurtscheller,et al.  Event-related synchronization (ERS) in the alpha band--an electrophysiological correlate of cortical idling: a review. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[2]  U. Raghavendra,et al.  A deep learning approach for Parkinson’s disease diagnosis from EEG signals , 2018, Neural Computing and Applications.

[3]  Wolfgang Rosenstiel,et al.  Brain-computer interfaces for educational applications , 2017 .

[4]  Min Liu,et al.  A Deep Transfer Convolutional Neural Network Framework for EEG Signal Classification , 2019, IEEE Access.

[5]  Seungjin Choi,et al.  A hybrid NIRS-EEG system for self-paced brain computer interface with online motor imagery , 2015, Journal of Neuroscience Methods.

[6]  Aidong Zhang,et al.  A Novel Semi-Supervised Deep Learning Framework for Affective State Recognition on EEG Signals , 2014, 2014 IEEE International Conference on Bioinformatics and Bioengineering.

[7]  Wan-Young Chung,et al.  A Single-Channel SSVEP-Based BCI Speller Using Deep Learning , 2019, IEEE Access.

[8]  H. Adeli,et al.  Analysis of EEG records in an epileptic patient using wavelet transform , 2003, Journal of Neuroscience Methods.

[9]  Hiroshi Ishiguro,et al.  A Non-parametric Approach to the Overall Estimate of Cognitive Load Using NIRS Time Series , 2017, Front. Hum. Neurosci..

[10]  Yang Li,et al.  An Amplitudes-Perturbation Data Augmentation Method in Convolutional Neural Networks for EEG Decoding , 2018, 2018 5th International Conference on Information, Cybernetics, and Computational Social Systems (ICCSS).

[11]  Robert Oostenveld,et al.  Estimating workload using EEG spectral power and ERPs in the n-back task , 2012, Journal of neural engineering.

[12]  Febo Cincotti,et al.  Hybrid P300-based brain-computer interface to improve usability for people with severe motor disability: electromyographic signals for error correction during a spelling task. , 2015, Archives of physical medicine and rehabilitation.

[13]  Andrzej Cichocki,et al.  Modern Machine-Learning Algorithms: For Classifying Cognitive and Affective States From Electroencephalography Signals , 2020, IEEE Systems, Man, and Cybernetics Magazine.

[14]  Michalis E. Zervakis,et al.  A Long Short-Term Memory deep learning network for the prediction of epileptic seizures using EEG signals , 2018, Comput. Biol. Medicine.

[15]  Carole R. Beal,et al.  EEG estimates of engagement and cognitive workload predict math problem solving outcomes , 2012, UMAP.

[16]  Feng Duan,et al.  A Novel Deep Learning Approach With Data Augmentation to Classify Motor Imagery Signals , 2019, IEEE Access.

[17]  Min Wang,et al.  Augmenting The Size of EEG datasets Using Generative Adversarial Networks , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[18]  Guang-Zhong Yang,et al.  Data augmentation for self-paced motor imagery classification with C-LSTM , 2019, Journal of neural engineering.

[19]  Ekapol Chuangsuwanich,et al.  Universal Joint Feature Extraction for P300 EEG Classification Using Multi-Task Autoencoder , 2018, IEEE Access.

[20]  Robert J. K. Jacob,et al.  Combining Electroencephalograph and Functional Near Infrared Spectroscopy to Explore Users' Mental Workload , 2009, HCI.

[21]  Qing-guo Ma,et al.  Mental Workload Analysis during the Production Process: EEG and GSR Activity , 2012 .

[22]  Theerawit Wilaiprasitporn,et al.  Affective EEG-Based Person Identification Using the Deep Learning Approach , 2018, IEEE Transactions on Cognitive and Developmental Systems.

[23]  Ah Chung Tsoi,et al.  Classification of EEG signals using the wavelet transform , 1997, Proceedings of 13th International Conference on Digital Signal Processing.

[24]  Jürgen Schmidhuber,et al.  Applying LSTM to Time Series Predictable through Time-Window Approaches , 2000, ICANN.

[25]  Xingyu Wang,et al.  Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI , 2019, IEEE Transactions on Cybernetics.

[26]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[27]  S. Murugappan,et al.  Human emotion recognition through short time Electroencephalogram (EEG) signals using Fast Fourier Transform (FFT) , 2013, 2013 IEEE 9th International Colloquium on Signal Processing and its Applications.

[28]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[29]  Rupert Ortner,et al.  A New Method to Generate Artificial Frames Using the Empirical Mode Decomposition for an EEG-Based Motor Imagery BCI , 2018, Front. Neurosci..

[30]  Thomas Dierks,et al.  BOLD correlates of EEG alpha phase-locking and the fMRI default mode network , 2009, NeuroImage.

[31]  Brent Lance,et al.  EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces , 2016, Journal of neural engineering.

[32]  Subhrajit Roy,et al.  ChronoNet: A Deep Recurrent Neural Network for Abnormal EEG Identification , 2018, AIME.

[33]  Yantao Yu,et al.  Automatic Biomechanical Workload Estimation for Construction Workers by Computer Vision and Smart Insoles , 2019 .

[34]  Maarten A. Hogervorst,et al.  Are You Really Looking? Finding the Answer through Fixation Patterns and EEG , 2009, HCI.

[35]  Kemal Polat,et al.  Classification of epileptiform EEG using a hybrid system based on decision tree classifier and fast Fourier transform , 2007, Appl. Math. Comput..

[36]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[37]  Alan F. Smeaton,et al.  Use of Neural Signals to Evaluate the Quality of Generative Adversarial Network Performance in Facial Image Generation , 2018, Cognitive Computation.

[38]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[39]  Hojjat Adeli,et al.  A Wavelet-Chaos Methodology for Analysis of EEGs and EEG Subbands to Detect Seizure and Epilepsy , 2007, IEEE Transactions on Biomedical Engineering.

[40]  Yinjing Guo,et al.  EEG-based Safety Driving Performance Estimation and Alertness Using Support Vector Machine , 2015 .

[41]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[42]  Matthias Rötting,et al.  Influence of Task Combination on EEG Spectrum Modulation for Driver Workload Estimation , 2011, Hum. Factors.

[43]  Theerawit Wilaiprasitporn,et al.  Consumer Grade EEG Measuring Sensors as Research Tools: A Review , 2020, IEEE Sensors Journal.

[44]  G Schneider,et al.  Eye tracking for assessment of workload: a pilot study in an anaesthesia simulator environment. , 2011, British journal of anaesthesia.

[45]  Toby P. Breckon,et al.  On the Classification of SSVEP-Based Dry-EEG Signals via Convolutional Neural Networks , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[46]  P. Geethanjali,et al.  DWT Based Detection of Epileptic Seizure From EEG Signals Using Naive Bayes and k-NN Classifiers , 2016, IEEE Access.

[47]  Shaolin Zhu,et al.  Dynamic Displacement Forecasting of Dashuitian Landslide in China Using Variational Mode Decomposition and Stack Long Short-Term Memory Network , 2019, Applied Sciences.

[48]  Fernando Seoane,et al.  Evaluation of physiological workload assessment methods using heart rate and accelerometry for a smart wearable system , 2019, Ergonomics.

[49]  Anne-Marie Brouwer,et al.  Measuring workload using a combination of electroencephalography and near infrared spectroscopy , 2012 .

[50]  N. T. Dzung.  Multifractality in NIRS Data of Brain Activity , 2010 .

[51]  Guanghua Xu,et al.  Using Corticomuscular Coherence to Reflect Function Recovery of Paretic Upper Limb after Stroke: A Case Study , 2018, Front. Neurol..

[52]  Xingyu Wang,et al.  Frequency Recognition in SSVEP-Based BCI using Multiset Canonical Correlation Analysis , 2013, Int. J. Neural Syst..