Adaptive confidence learning for the personalization of pain intensity estimation systems

In this work, a method is presented for the continuous estimation of pain intensity based on fusion of bio-physiological and video features. Furthermore, a method is proposed for the adaptation of the system to unknown test persons based on unlabeled data. First, an analysis is presented that shows which modalities and feature sets are suited best for the task of recognizing pain levels in a person-independent setting. For this, a large set of features is extracted from the available bio-physiological channels (ECG, EMG and skin conductivity) and the video stream. We then propose a method to learn the confidence of a regression system using a multi-stage ensemble classifier. Based on the outcome of the classifier, which is realized by a neural network, confident samples are selected by the adaptation procedure. In various experiments, we show that the algorithm is able to detect highly confident samples which can be used to improve the overall performance. We furthermore discuss the current limitations of automatic pain intensity estimation—in light of the presented approach and beyond.

[1]  Semyon Slobounov,et al.  Application of a novel measure of EEG non-stationarity as ‘Shannon- entropy of the peak frequency shifting’ for detecting residual abnormalities in concussed individuals , 2011, Clinical Neurophysiology.

[2]  Zhengyou Zhang,et al.  Taylor expansion based classifier adaptation: Application to person detection , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Maja Pantic,et al.  Continuous Pain Intensity Estimation from Facial Expressions , 2012, ISVC.

[4]  Dennis C. Tkach,et al.  Study of stability of time-domain features for electromyographic pattern recognition , 2010, Journal of NeuroEngineering and Rehabilitation.

[5]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[6]  Carlos Busso,et al.  Supervised domain adaptation for emotion recognition from speech , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Georg Dorffner,et al.  ADAPTIVE MACHINE LEARNING IN DELAYED FEEDBACK DOMAINS BY SELECTIVE RELEARNING , 2008, Appl. Artif. Intell..

[8]  Rong Yan,et al.  Cross-domain video concept detection using adaptive svms , 2007, ACM Multimedia.

[9]  Hichem Sahli,et al.  Adaptive Real-Time Emotion Recognition from Body Movements , 2016, TIIS.

[10]  Tsuhan Chen,et al.  The painful face - Pain expression recognition using active appearance models , 2009, Image Vis. Comput..

[11]  Patrick Thiam,et al.  Multimodal Data Fusion for Person-Independent, Continuous Estimation of Pain Intensity , 2015, EANN.

[12]  Ayoub Al-Hamadi,et al.  Automatic Pain Recognition from Video and Biomedical Signals , 2014, 2014 22nd International Conference on Pattern Recognition.

[13]  Eric Duviella,et al.  An evolving classification approach for fault diagnosis and prognosis of a wind farm , 2013, 2013 Conference on Control and Fault-Tolerant Systems (SysTol).

[14]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Gustavo Moreira da Silva,et al.  Automatic pain quantification using autonomic parameters , 2014 .

[16]  Tanu Sharma,et al.  A novel feature extraction for robust EMG pattern recognition , 2016, Journal of medical engineering & technology.

[17]  M. Benedek,et al.  Decomposition of skin conductance data by means of nonnegative deconvolution , 2010, Psychophysiology.

[18]  Liqing Zhang,et al.  ECG Feature Extraction and Classification Using Wavelet Transform and Support Vector Machines , 2005, 2005 International Conference on Neural Networks and Brain.

[19]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[20]  Davide Fossati,et al.  Affect detection from non-stationary physiological data using ensemble classifiers , 2015, Evol. Syst..

[21]  Sascha Meudt,et al.  Fusion of Audio-visual Features using Hierarchical Classifier Systems for the Recognition of Affective States and the State of Depression , 2014, ICPRAM.

[22]  Björn W. Schuller,et al.  LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework , 2013, Image Vis. Comput..

[23]  J. Allwood.  A Framework for Studying Human Multimodal Communication , 2013 .

[24]  Dongmei Jiang,et al.  Multimodal Affective Dimension Prediction Using Deep Bidirectional Long Short-Term Memory Recurrent Neural Networks , 2015, AVEC@ACM Multimedia.

[25]  Omar AlZoubi,et al.  Classification of EEG for Affect Recognition: An Adaptive Approach , 2009, Australasian Conference on Artificial Intelligence.

[26]  Patrick Thiam,et al.  Methods for Person-Centered Continuous Pain Intensity Assessment From Bio-Physiological Channels , 2016, IEEE Journal of Selected Topics in Signal Processing.

[27]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[28]  Weiting Chen,et al.  Measuring complexity using FuzzyEn, ApEn, and SampEn. , 2009, Medical engineering & physics.

[29]  Jeffrey F. Cohn,et al.  Automatic detection of pain intensity , 2012, ICMI '12.

[30]  R. Treister,et al.  Differentiating between heat pain intensities: The combined effect of multiple autonomic parameters , 2012, PAIN®.

[31]  Sascha Meudt,et al.  Revisiting the EmotiW challenge: how wild is it really? , 2015, Journal on Multimodal User Interfaces.

[32]  Markus Kächele,et al.  Inferring Depression and Affect from Application Dependent Meta Knowledge , 2014, AVEC '14.

[33]  Panagiotis K. Artemiadis,et al.  An EMG-Based Robot Control Scheme Robust to Time-Varying EMG Signal Features , 2010, IEEE Transactions on Information Technology in Biomedicine.

[34]  Ayoub Al-Hamadi,et al.  The biovid heat pain database data for the advancement and systematic validation of an automated pain recognition system , 2013, 2013 IEEE International Conference on Cybernetics (CYBCO).

[35]  Jeffrey F. Cohn,et al.  Painful data: The UNBC-McMaster shoulder pain expression archive database , 2011, Face and Gesture 2011.

[36]  Patrick Thiam,et al.  Ensemble Methods for Continuous Affect Recognition: Multi-modality, Temporality, and Challenges , 2015, AVEC@ACM Multimedia.

[37]  Markus Kächele,et al.  Multiple Classifier Systems for the Classification of Audio-Visual Emotional States , 2011, ACII.

[38]  Markus Kächele,et al.  Bio-Visual Fusion for Person-Independent Recognition of Pain Intensity , 2015, MCS.

[39]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.