Emotional empathy transition patterns from human brain responses in interactive communication situations

The paper reports our research aiming at utilization of human interactive communication modeling principles in application to a novel interaction paradigm designed for brain–computer/machine-interfacing (BCI/BMI) technologies as well as for socially aware intelligent environments or communication support systems. Automatic procedures for human affective responses or emotional states estimation are still a hot topic of contemporary research. We propose to utilize human brain and bodily physiological responses for affective/emotional as well as communicative interactivity estimation, which potentially could be used in the future for human–machine/environment interaction design. As a test platform for such an intelligent human–machine communication application, an emotional stimuli paradigm was chosen to evaluate brain responses to various affective stimuli in an emotional empathy mode. Videos with moving faces expressing various emotional displays as well as speech stimuli with similarly emotionally articulated sentences are presented to the subjects in order to further analyze different affective responses. From information processing point of view, several challenges with multimodal signal conditioning and stimuli dynamic response extraction in time frequency domain are addressed. Emotions play an important role in human daily life and human-to-human communication. This is why involvement of affective stimuli principles to human–machine communication or machine-mediated communication with utilization of multichannel neurophysiological and periphery physiological signals monitoring techniques, allowing real-time subjective brain responses evaluation, is discussed. We present our preliminary results and discuss potential applications of brain/body affective responses estimation for future interactive/smart environments.

[1]  Danilo P. Mandic,et al.  Artificial Intelligence for Human Computing , 2007 .

[2]  Dennis J. McFarland,et al.  Brain-Computer Interface Operation of Robotic and Prosthetic Devices , 2008, Computer.

[3]  Ernst Fernando Lopes Da Silva Niedermeyer,et al.  Electroencephalography, basic principles, clinical applications, and related fields , 1982 .

[4]  Wade Junek,et al.  Mind Reading: The Interactive Guide to Emotions , 2007 .

[5]  Danilo P. Mandic,et al.  Modelling the Communication Atmosphere: A Human Centered Multimedia Approach to Evaluate Communicative Situations , 2007, Artifical Intelligence for Human Computing.

[6]  Christa Neuper,et al.  Rehabilitation with Brain-Computer Interface Systems , 2008, Computer.

[7]  Enhua Shen,et al.  Advances in Cognitive Neurodynamics ICCN 2007: Proceedings of the International Conference on Cognitive Neurodynamics. ICCN 2007 Proceedings , 2008 .

[8]  Georgios C. Anagnostopoulos,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2003, Lecture Notes in Computer Science.

[9]  E. Sellers,et al.  How many people are able to control a P300-based brain–computer interface (BCI)? , 2009, Neuroscience Letters.

[10]  Toshihisa Tanaka,et al.  Multichannel spectral pattern separation - An EEG processing application - , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Allan Kardec Barros,et al.  A Multimodal Approach to Communicative Interactivity Classification , 2007, J. VLSI Signal Process..

[12]  Clemens Brunner,et al.  BioSig: A Free and Open Source Software Library for BCI Research , 2008, Computer.

[13]  Danilo P. Mandic,et al.  Engineering Approaches to Conversational Informatics , 2008 .

[14]  Yoko Yamakata,et al.  Toward the Human Communication Efficiency Monitoring from Captured Audio and Video Media in Real Environments , 2003, KES.

[15]  Michitaka Hirose,et al.  Brain-Computer Interfaces, Virtual Reality, and Videogames , 2008, Computer.

[16]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[17]  Andrzej Cichocki,et al.  Emotional States Estimation from Multichannel EEG Maps , 2008 .

[18]  Liqing Zhang,et al.  Noninvasive BCIs: Multiway Signal-Processing Array Decompositions , 2008, Computer.

[19]  Andrzej Cichocki,et al.  Emd Approach to Multichannel EEG Data - the amplitude and Phase Components Clustering Analysis , 2010, J. Circuits Syst. Comput..