An Affective BCI Using Multiple ERP Components Associated to Facial Emotion Processing

P300-based brain computer interfaces (BCIs) have successfully demonstrated that attention to an oddball stimulus can enhance the P300 component of the event-related potential (ERP) phase-locked to the event. However, it was unclear whether the more sophisticated face-evoked potentials can also be modulated by related mental tasks under the oddball paradigm. This study investigated ERP responses to image stimuli of objects, neutral faces, and emotional faces when subjects perform attention, face recognition and discrimination of emotional facial expressions respectively under the oddball paradigm. The results revealed the significant difference between target and non-target ERPs for each mental task. In addition, significant differences among the three mental tasks were observed for vertex-positive potential (VPP) over the fronto-central sites, late positive potential (LPP)/P3b over the centro-parietal sites and N250 over the occipito-temporal sites. These findings indicate that a novel affective BCI paradigm can be developed based on detection of multiple ERP components reflecting human face encoding and emotion processing. The high classification performance for single-trial emotional face-related ERPs demonstrated the effectiveness of the affective BCI.

[1]  Liqing Zhang,et al.  A Novel Oddball Paradigm for Affective BCIs Using Emotional Faces as Stimuli , 2011, ICONIP.

[2]  A. Burton,et al.  N250r: a face-selective brain response to stimulus repetitions , 2004, Neuroreport.

[3]  Genevieve McArthur,et al.  Switching associations between facial identity and emotional expression: A behavioural and ERP study , 2010, NeuroImage.

[4]  Dirk Heylen,et al.  Affective Brain-Computer Interfaces (aBCI 2011) , 2011, ACII.

[5]  D. Yao,et al.  An Enhanced Probabilistic LDA for Multi-Class Brain Computer Interface , 2011, PloS one.

[6]  Scott Makeig,et al.  First Demonstration of a Musical Emotion BCI , 2011, ACII.

[7]  Margot J. Taylor,et al.  N170 or N1? Spatiotemporal differences between object and face processing using ERPs. , 2004, Cerebral cortex.

[8]  A. Kübler,et al.  Flashing characters with famous faces improves ERP-based brain–computer interface performance , 2011, Journal of neural engineering.

[9]  Benjamin Blankertz,et al.  A novel brain-computer interface based on the rapid serial visual presentation paradigm , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[10]  S M M Martens,et al.  Overlap and refractory effects in a brain–computer interface speller based on the visual P300 event-related potential , 2009, Journal of neural engineering.

[11]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[12]  Tracy A. Dennis,et al.  Emotional face processing and attention performance in three domains: neurophysiological mechanisms and moderating effects of trait anxiety. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[13]  Yang Seok Cho,et al.  Neural correlates of top–down processing in emotion perception: An ERP study of emotional faces in white noise versus noise-alone stimuli , 2010, Brain Research.

[14]  B. Blankertz,et al.  (C)overt attention and visual speller design in an ERP-based brain-computer interface , 2010, Behavioral and Brain Functions.

[15]  Hossein Esteky,et al.  A study of N250 event-related brain potential during face and non-face detection tasks. , 2009, Journal of vision.

[16]  Margot J. Taylor,et al.  Inversion and Contrast Polarity Reversal Affect both Encoding and Recognition Processes of Unfamiliar Faces: A Repetition Study Using ERPs , 2002, NeuroImage.

[17]  Wenfeng Feng,et al.  Three stages of facial expression processing: ERP study with rapid serial visual presentation , 2010, NeuroImage.

[18]  Maria Pia Viggiano,et al.  Interplay between familiarity and orientation in face processing: an ERP study. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[19]  N. Birbaumer,et al.  An auditory oddball (P300) spelling system for brain-computer interfaces. , 2009, Psychophysiology.

[20]  V. Goffaux,et al.  Spatio-temporal localization of the face inversion effect: an event-related potentials study , 1999, Biological Psychology.

[21]  Stefan Robert Schweinberger,et al.  Face and object encoding under perceptual load: ERP evidence , 2011, NeuroImage.

[22]  A. Cichocki,et al.  A novel BCI based on ERP components sensitive to configural processing of human faces , 2012, Journal of neural engineering.

[23]  M. Thulasidas,et al.  Robust classification of EEG signal for brain-computer interface , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  A. Lenhardt,et al.  An Adaptive P300-Based Online Brain–Computer Interface , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[25]  Xingyu Wang,et al.  A combined brain–computer interface based on P300 potentials and motion-onset visual evoked potentials , 2012, Journal of Neuroscience Methods.

[26]  Talma Hendler,et al.  The validity of the face-selective ERP N170 component during simultaneous recording with functional MRI , 2008, NeuroImage.

[27]  G. Winocur,et al.  What Is Special about Face Recognition? Nineteen Experiments on a Person with Visual Object Agnosia and Dyslexia but Normal Face Recognition , 1997, Journal of Cognitive Neuroscience.

[28]  Peter J. Disimile,et al.  Vortex entrainment and separation using flow superposition , 2006, J. Vis..

[29]  Xiaorong Gao,et al.  A brain–computer interface using motion-onset visual evoked potential , 2008, Journal of neural engineering.

[30]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[31]  M. Eimer.  Effects of face inversion on the structural encoding and recognition of faces. Evidence from event-related brain potentials. , 2000, Brain research. Cognitive brain research.

[32]  J. Wolpaw,et al.  A novel P300-based brain–computer interface stimulus presentation paradigm: Moving beyond rows and columns , 2010, Clinical Neurophysiology.