Mutual information based feature subset selection in multivariate time series classification

Abstract This paper deals with supervised classification of multivariate time series. In particular, the goal is to propose a filter method to select a subset of time series. Consequently, we adopt the framework proposed by Brown et al. [1]. The key point in this framework is the computation of the mutual information between the features, which allows us to measure the relevance of each feature subset. In our case, where the features are a time series, we use an adaptation of existing nonparametric mutual information estimators based on the k-nearest neighbor. Specifically, for the purpose of bringing these methods to the time series scenario, we rely on the use of dynamic time warping dissimilarity. Our experimental results show that our method is able to strongly reduce the number of time series while keeping or increasing the classification accuracy.

[1]  Thomas Philip Runarsson,et al.  Test-retest reliability and feature selection in physiological time series classification , 2012, Comput. Methods Programs Biomed..

[2]  C. Baker.  Mutual Information for Gaussian Processes , 1970 .

[3]  Eamonn J. Keogh,et al.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances , 2016, Data Mining and Knowledge Discovery.

[4]  Michel Verleysen,et al.  A Mutual Information estimator for continuous and discrete variables applied to Feature Selection and Classification problems , 2016, Int. J. Comput. Intell. Syst..

[5]  Javier Del Ser,et al.  Nature-inspired approaches for distance metric learning in multivariate time series classification , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[6]  Edwin R. Hancock,et al.  Adaptive Feature Selection Based on the Most Informative Graph-Based Features , 2017, GbRPR.

[7]  Mustafa Gokce Baydogan,et al.  Autoregressive forests for multivariate time series modeling , 2018, Pattern Recognit..

[8]  Min Han,et al.  Joint mutual information-based input variable selection for multivariate time series modeling , 2015, Eng. Appl. Artif. Intell..

[9]  Qin Zhang,et al.  Time series feature learning with labeled and unlabeled data , 2019, Pattern Recognit..

[10]  Marc Boullé,et al.  FEARS: a Feature and Representation Selection approach for Time Series Classification , 2019, ACML.

[11]  Jun Wang,et al.  Generalizing DTW to the multi-dimensional case requires an adaptive approach , 2016, Data Mining and Knowledge Discovery.

[12]  Michael Flynn,et al.  The UEA multivariate time series classification archive, 2018 , 2018, ArXiv.

[13]  Claudio De Stefano,et al.  A ranking-based feature selection approach for handwritten character recognition , 2019, Pattern Recognit. Lett..

[14]  Haikun Wei,et al.  Mutual information based feature selection for multivariate time series forecasting , 2016, CCC 2016.

[15]  Timothy A. Gonsalves,et al.  A joint feature selection framework for multivariate resource usage prediction in cloud servers using stability and prediction performance , 2018, The Journal of Supercomputing.

[16]  Bing Xue,et al.  Mutual Information Estimation for Filter Based Feature Selection Using Particle Swarm Optimization , 2016, EvoApplications.

[17]  Cyrus Shahabi,et al.  Feature subset selection and feature ranking for multivariate time series , 2005, IEEE Transactions on Knowledge and Data Engineering.

[18]  Claude E. Shannon,et al.  The mathematical theory of communication , 1950 .

[19]  Oksam Chae,et al.  Simultaneous feature selection and discretization based on mutual information , 2019, Pattern Recognit..

[20]  Friedrich Schmid,et al.  Mutual information as a measure of multivariate association: analytical properties and statistical estimation , 2012 .

[21]  Rodica Potolea,et al.  Time series — A taxonomy based survey , 2017, 2017 13th IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).

[22]  Antonio F. Gómez-Skarmeta,et al.  A methodology for energy multivariate time series forecasting in smart buildings based on feature selection , 2019, Energy and Buildings.

[23]  S Roberts,et al.  Gaussian processes for time-series modelling , 2013, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[24]  Vadim V. Strijov,et al.  Multi-way feature selection for ECoG-based Brain-Computer Interface , 2018, Expert Syst. Appl..

[25]  A. Arifin,et al.  Correlation and Symmetrical Uncertainty-Based Feature Selection for Multivariate Time Series Classification , 2019, International Journal of Intelligent Engineering and Systems.

[26]  Xiaoying Wu,et al.  An ensemble of shapelet-based classifiers on inter-class and intra-class imbalanced multivariate time series at the early stage , 2018, Soft Comput..

[27]  Chunlin Chen,et al.  Univariate time series classification using information geometry , 2019, Pattern Recognit..

[28]  José Antonio Lozano,et al.  A review on distance based time series classification , 2018, Data Mining and Knowledge Discovery.

[29]  Bing Xue,et al.  Mutual information for feature selection: estimation or counting? , 2016, Evol. Intell..

[30]  Alan Jovic,et al.  Classification of cardiac arrhythmias based on alphabet entropy of heart rate variability time series , 2017, Biomed. Signal Process. Control..

[31]  Yuming Zhou,et al.  A Feature Subset Selection Algorithm Automatic Recommendation Method , 2013, J. Artif. Intell. Res..

[32]  B. Chakraborty.  Feature Selection and Classification Techniques for Multivariate Time Series , 2007, Second International Conference on Innovative Computing, Informatio and Control (ICICIC 2007).

[33]  Gavin Brown,et al.  Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection , 2012, J. Mach. Learn. Res..

[34]  Brian C. Ross.  Mutual Information between Discrete and Continuous Data Sets , 2014, PloS one.

[35]  Arvind Kumar Shekar,et al.  Selection of Relevant and Non-Redundant Multivariate Ordinal Patterns for Time Series Classification , 2018, DS.

[36]  Javier Del Ser,et al.  On-line Elastic Similarity Measures for time series , 2019, Pattern Recognit..

[37]  Klemens Böhm,et al.  Iterative Estimation of Mutual Information with Error Bounds , 2019, EDBT.

[38]  A. Bulinski,et al.  Statistical estimation of conditional Shannon entropy , 2018, ESAIM: Probability and Statistics.

[39]  Johan A. K. Suykens,et al.  Transductive Feature Selection Using Clustering-Based Sample Entropy for Temperature Prediction in Weather Forecasting , 2018, Entropy.

[40]  Mingwei Yu,et al.  Feature selection method based on mutual information and class separability for dimension reduction in multidimensional time series for clinical data , 2015, Biomed. Signal Process. Control..

[41]  Erwin Prassler,et al.  Applicability of feature selection on multivariate time series data for robotic discovery , 2010, 2010 3rd International Conference on Advanced Computer Theory and Engineering(ICACTE).

[42]  Pablo A. Estévez,et al.  A review of feature selection methods based on mutual information , 2015, Neural Computing and Applications.

[43]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.