Modern Machine-Learning Algorithms: For Classifying Cognitive and Affective States From Electroencephalography Signals

Estimating cognitive or affective states from brain signals is a key but challenging step in creating passive brain-computer interface (BCI) applications. So far, estimating mental workloads or emotions from electroencephalogram (EEG) signals is only feasible with modest classification accuracies, which thus lead to unreliable neuroadaptive applications. However, recent machine-learning algorithms, notably Riemannian geometry-based classifiers (RGCs) and convolutional neural networks (CNNs), have shown promise for other BCI systems, e.g., motor imagery BCIs. However, they have not been formally studied and compared for cognitive or affective states classification.

[1]  Desney S. Tan,et al.  Feasibility and pragmatics of classifying working memory load with an electroencephalograph , 2008, CHI.

[2]  Robert J. K. Jacob,et al.  Using fNIRS brain sensing to evaluate information visualization interfaces , 2013, CHI.

[3]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Nigel H. Lovell,et al.  Estimating cognitive workload using wavelet entropy-based features during an arithmetic task , 2013, Comput. Biol. Medicine.

[5]  S. Fairclough.  BCI and Physiological Computing for Computer Games : Differences , Similarities & Intuitive Control , 2009 .

[6]  Jérémy Frey,et al.  Framework for Electroencephalography-based Evaluation of User Experience , 2016, CHI.

[7]  Cuntai Guan,et al.  Filter Bank Common Spatial Pattern Algorithm on BCI Competition IV Datasets 2a and 2b , 2012, Front. Neurosci..

[8]  Hasan Ayaz,et al.  NEUROERGONOMICS , 2021, Handbook of Human Factors and Ergonomics.

[9]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[10]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[11]  K.-R. Muller,et al.  Optimizing Spatial filters for Robust EEG Single-Trial Analysis , 2008, IEEE Signal Processing Magazine.

[12]  Abeer Al-Nafjan,et al.  Recognition of Affective States via Electroencephalogram Analysis and Classification , 2018, IHSI.

[13]  Charalampos Bratsas,et al.  Toward Emotion Aware Computing: An Integrated Approach Using Multichannel Neurophysiological Recordings and Affective Visual Stimuli , 2010, IEEE Transactions on Information Technology in Biomedicine.

[14]  Christian Jutten,et al.  Riemannian Geometry Applied to BCI Classification , 2010, LVA/ICA.

[15]  Alexandre Barachant,et al.  Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review , 2017 .

[16]  A. Nijholt,et al.  A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges , 2014 .

[17]  Andrzej Cichocki,et al.  EmotionMeter: A Multimodal Framework for Recognizing Human Emotions , 2019, IEEE Transactions on Cybernetics.

[18]  Guillaume Chanel,et al.  Emotion Assessment From Physiological Signals for Adaptation of Game Difficulty , 2011, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[19]  F. Lotte,et al.  EEG-based neuroergonomics for 3d user interfaces: opportunities and challenges , 2017 .

[20]  Andrzej Cichocki,et al.  Towards Robust Neuroadaptive HCI: Exploring Modern Machine Learning Methods to Estimate Mental Workload From EEG Signals , 2018, CHI Extended Abstracts.

[21]  Srinivasan Jayaraman,et al.  Electroencephalography Amplitude Modulation Analysis for Automated Affective Tagging of Music Video Clips , 2018, Front. Comput. Neurosci..

[22]  Nilanjan Sarkar,et al.  Maintaining Optimal Challenge in Computer Games through Real-Time Physiological Feedback , 2005 .

[23]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[24]  Thibault Gateau,et al.  Real-Time State Estimation in a Flight Simulator Using fNIRS , 2015, PloS one.

[25]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[26]  Christian Mühl,et al.  EEG-based workload estimation across affective contexts , 2014, Front. Neurosci..

[27]  Karen Dijkstra,et al.  Turning negative into positives! Exploiting ‘negative’ results in Brain–Machine Interface (BMI) research , 2019, Brain-Computer Interfaces.

[28]  Hasan Al-Nashash,et al.  Novel Classification System for Classifying Cognitive Workload Levels Under Vague Visual Stimulation , 2017, IEEE Sensors Journal.

[29]  Christian Mühl,et al.  Valence, arousal and dominance in the EEG during game play , 2013, Int. J. Auton. Adapt. Commun. Syst..

[30]  Robert Oostenveld,et al.  Estimating workload using EEG spectral power and ERPs in the n-back task , 2012, Journal of neural engineering.

[31]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[32]  Christian Kothe,et al.  Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general , 2011, Journal of neural engineering.

[33]  Hung T. Nguyen,et al.  EEG emotion recognition using reduced channel wavelet entropy and average wavelet coefficient features with normal Mutual Information method , 2017, 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[34]  Christian Jutten,et al.  Multiclass Brain–Computer Interface Classification by Riemannian Geometry , 2012, IEEE Transactions on Biomedical Engineering.

[35]  Stephen H. Fairclough,et al.  Fundamentals of physiological computing , 2009, Interact. Comput..

[36]  F. Yger,et al.  Riemannian Approaches in Brain-Computer Interfaces: A Review , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[37]  C. Neuper,et al.  Combining Brain–Computer Interfaces and Assistive Technologies: State-of-the-Art and Challenges , 2010, Front. Neurosci..

[38]  J. Russell.  A circumplex model of affect. , 1980 .

[39]  Daniel Afergan,et al.  Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State , 2016, CHI.

[40]  Scott Makeig,et al.  Emotion Recognition from EEG during Self-Paced Emotional Imagery , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[41]  Karim Jerbi,et al.  Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy , 2015, Journal of Neuroscience Methods.