GFIL: A Unified Framework for the Importance Analysis of Features, Frequency Bands, and Channels in EEG-Based Emotion Recognition

Accurately and automatically recognizing the emotional states of human beings is the central task in affective computing. The electroencephalography (EEG) data, generated from the neural activities in brain cortex, provide us with a reliable data source to perform emotion recognition. Besides the recognition accuracy, it is also necessary to explore the importance of different EEG features, frequency bands, and channels in emotion expression. In this article, we propose a unified framework termed graph-regularized least square regression with feature importance learning (GFIL) to simultaneously achieve these goals by incorporating an autoweighting variable into the least square regression. Unlike the widely used trial-and-error manner, GFIL automatically completes the identification once it is trained. Specifically, GFIL can: 1) adaptively discriminate the contributions of different feature dimensions; 2) automatically identify the critical frequency bands and channels; and 3) quantitatively rank and select the features by the learned autoweighting variable. From the experimental results on the SEED_IV data set, we find GFIL obtained improved accuracies based on the feature autoweighting strategy, which are 75.33%, 75.03%, and 79.17% corresponding to the three cross-session recognition tasks (session1->session2, session1->session3, session2->session3), respectively. Additionally, the Gamma band is identified as the most important one and the channels locating in the prefrontal and left/right central regions are more important.

[1]  Wanzeng Kong,et al.  A joint optimization framework to semi-supervised RVFL and ELM networks for efficient data classification , 2020, Appl. Soft Comput..

[2]  Wenming Zheng,et al.  EEG Emotion Recognition Using Dynamical Graph Convolutional Neural Networks , 2020, IEEE Transactions on Affective Computing.

[3]  Yong Peng,et al.  Joint Semi-Supervised Feature Auto-Weighting and Classification Model for EEG-Based Cross-Subject Sleep Quality Evaluation , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Yifan Xu,et al.  Transfer Learning for EEG-Based Brain–Computer Interfaces: A Review of Progress Made Since 2016 , 2020, IEEE Transactions on Cognitive and Developmental Systems.

[5]  Mohammad Hassan Moradi,et al.  FSIFT-PLV: An emerging phase synchrony index , 2020, Biomed. Signal Process. Control..

[6]  Feiping Nie,et al.  Semi-Supervised Feature Selection via Sparse Rescaled Linear Square Regression , 2020, IEEE Transactions on Knowledge and Data Engineering.

[7]  A. Goshvarpour,et al.  A Novel Approach for EEG Electrode Selection in Automated Emotion Recognition Based on Lagged Poincare’s Indices and sLORETA , 2019, Cognitive Computation.

[8]  Hui Song,et al.  Channel Selection Method for EEG Emotion Recognition Using Normalized Mutual Information , 2019, IEEE Access.

[9]  Fabio Babiloni,et al.  EEG Fingerprints: Phase Synchronization of EEG Signals as Biomarker for Subject Identification , 2019, IEEE Access.

[10]  Andrzej Cichocki,et al.  EmotionMeter: A Multimodal Framework for Recognizing Human Emotions , 2019, IEEE Transactions on Cybernetics.

[11]  Gernot R. Müller-Putz,et al.  Domain Adaptation Techniques for EEG-Based Emotion Recognition: A Comparative Study on Two Public Datasets , 2019, IEEE Transactions on Cognitive and Developmental Systems.

[12]  Yongtian He,et al.  Deep learning for electroencephalogram (EEG) classification tasks: a review , 2019, Journal of neural engineering.

[13]  M. Shamim Hossain,et al.  Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization , 2019, ACM Trans. Multim. Comput. Commun. Appl..

[14]  Ying Zeng,et al.  EEG Based Emotion Recognition by Combining Functional Connectivity Network and Local Activations , 2019, IEEE Transactions on Biomedical Engineering.

[15]  Feiping Nie,et al.  Semi-supervised Feature Selection via Rescaled Linear Regression , 2017, IJCAI.

[16]  Mehmet Siraç Özerdem,et al.  Emotion recognition based on EEG features in movie clips with channel selection , 2017, Brain Informatics.

[17]  James L. Coyle,et al.  Deep Belief Networks for Electroencephalography: A Review of Recent Contributions and Future Outlooks , 2017, IEEE Journal of Biomedical and Health Informatics.

[18]  Wenming Zheng,et al.  Multichannel EEG-Based Emotion Recognition via Group Sparse Canonical Correlation Analysis , 2017, IEEE Transactions on Cognitive and Developmental Systems.

[19]  Wolfram Burgard,et al.  Deep learning with convolutional neural networks for EEG decoding and visualization , 2017, Human brain mapping.

[20]  Xiang Li,et al.  Emotion recognition from multi-channel EEG data through Convolutional Recurrent Neural Network , 2016, 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[21]  Bao-Liang Lu,et al.  A multimodal approach to estimating vigilance using EEG and forehead EOG , 2016, Journal of neural engineering.

[22]  Yu Cao,et al.  ReliefF-Based EEG Sensor Selection Methods for Emotion Recognition , 2016, Sensors.

[23]  John Atkinson,et al.  Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers , 2016, Expert Syst. Appl..

[24]  Yong Peng,et al.  Discriminative manifold extreme learning machine and applications to image and EEG signal classification , 2016, Neurocomputing.

[25]  Bao-Liang Lu,et al.  Identifying Stable Patterns over Time for Emotion Recognition from EEG , 2016, IEEE Transactions on Affective Computing.

[26]  Fathi E. Abd El-Samie,et al.  A review of channel selection algorithms for EEG signal processing , 2015, EURASIP Journal on Advances in Signal Processing.

[27]  Hyo Jong Lee,et al.  Deep learninig of EEG signals for emotion recognition , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[28]  Bao-Liang Lu,et al.  Investigating Critical Frequency Bands and Channels for EEG-Based Emotion Recognition with Deep Neural Networks , 2015, IEEE Transactions on Autonomous Mental Development.

[29]  Xianzhong Long,et al.  Discriminative graph regularized extreme learning machine and its application to face recognition , 2015, Neurocomputing.

[30]  Yong Peng,et al.  EEG-based emotion classification using deep belief networks , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[31]  Martin Buss,et al.  Feature Extraction and Selection for Emotion Recognition from EEG , 2014, IEEE Transactions on Affective Computing.

[32]  Bao-Liang Lu,et al.  Differential entropy feature for EEG-based emotion classification , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[33]  Bao-Liang Lu,et al.  Differential entropy feature for EEG-based vigilance estimation , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[34]  Leontios J. Hadjileontiadis,et al.  Toward an EEG-Based Recognition of Music Liking Using Time-Frequency Analysis , 2012, IEEE Transactions on Biomedical Engineering.

[35]  Volkan Cevher,et al.  Sparse projections onto the simplex , 2012, ICML.

[36]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[37]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[38]  A. H. Sherman On Newton-Iterative Methods for the Solution of Systems of Nonlinear Equations , 1978 .

[39]  Mohammad Soleymani,et al.  Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection , 2016, IEEE Transactions on Affective Computing.