A review on distance based time series classification

Time series classification is an increasing research topic due to the vast amount of time series data that is being created over a wide variety of fields. The particularity of the data makes it a challenging task and different approaches have been taken, including the distance based approach. 1-NN has been a widely used method within distance based time series classification due to its simplicity but still good performance. However, its supremacy may be attributed to being able to use specific distances for time series within the classification process and not to the classifier itself. With the aim of exploiting these distances within more complex classifiers, new approaches have arisen in the past few years that are competitive or which outperform the 1-NN based approaches. In some cases, these new methods use the distance measure to transform the series into feature vectors, bridging the gap between time series and traditional classifiers. In other cases, the distances are employed to obtain a time series kernel and enable the use of kernel methods for time series classification. One of the main challenges is that a kernel function must be positive semi-definite, a matter that is also addressed within this review. The presented review includes a taxonomy of all those methods that aim to classify time series using a distance based approach, as well as a discussion of the strengths and weaknesses of each method.

[1]  Daphna Weinshall,et al.  Classification with Nonmetric Distances: Image Retrieval and Class Representation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Eamonn J. Keogh,et al.  Logical-shapelets: an expressive primitive for time series classification , 2011, KDD.

[3]  Bernard Haasdonk,et al.  Feature space interpretation of SVMs with indefinite kernels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  Eamonn J. Keogh,et al.  Fast Shapelets: A Scalable Algorithm for Discovering Time Series Shapelets , 2013, SDM.

[6]  Ting Wang,et al.  Kernel Sparse Representation-Based Classifier , 2012, IEEE Transactions on Signal Processing.

[7]  Isabelle Guyon,et al.  UNIPEN project of on-line data exchange and recognizer benchmarks , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[8]  Lars Schmidt-Thieme,et al.  Learning time-series shapelets , 2014, KDD.

[9]  ArcosJosep Ll.,et al.  An empirical evaluation of similarity measures for time series classification , 2014 .

[10]  S. Canu,et al.  Training Invariant Support Vector Machines using Selective Sampling , 2005 .

[11]  Robert P. W. Duin,et al.  A Generalized Kernel Approach to Dissimilarity-based Classification , 2002, J. Mach. Learn. Res..

[12]  Clu-istos Foutsos,et al.  Fast Subsequence Matching in Time-Series Databases , 1994 .

[13]  P. Marteau,et al.  Constructing Positive Definite Elastic Kernels with Application to Time Series Classification , 2018 .

[14]  Jason Lines,et al.  Time series classification with ensembles of elastic distance measures , 2015, Data Mining and Knowledge Discovery.

[15]  Paul Lukowicz,et al.  On general purpose time series similarity measures and their use as kernel functions in support vector machines , 2014, Inf. Sci..

[16]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[17]  Chotirat Ratanamahatana,et al.  An Enhanced Support Vector Machine for Faster Time Series Classification , 2016, ACIIDS.

[18]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[19]  Christos Faloutsos,et al.  Efficiently supporting ad hoc queries in large datasets of time sequences , 1997, SIGMOD '97.

[20]  Nuno Constantino Castro,et al.  Time Series Data Mining , 2009, Encyclopedia of Database Systems.

[21]  Olufemi A. Omitaomu,et al.  Weighted dynamic time warping for time series classification , 2011, Pattern Recognit..

[22]  Eamonn J. Keogh,et al.  Reliable early classification of time series based on discriminating the classes over time , 2016, Data Mining and Knowledge Discovery.

[23]  Sylvie Gibet,et al.  On Recursive Edit Distance Kernels With Application to Time Series Classification , 2010, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Donald J. Berndt,et al.  Using Dynamic Time Warping to Find Patterns in Time Series , 1994, KDD Workshop.

[25]  Volkmar Frinken,et al.  Efficient temporal pattern recognition by means of dissimilarity space embedding with discriminative prototypes , 2017, Pattern Recognit..

[26]  Li Wei,et al.  Experiencing SAX: a novel symbolic representation of time series , 2007, Data Mining and Knowledge Discovery.

[27]  Pradeep Ravikumar,et al.  D2KE: From Distance to Kernel and Embedding , 2018, ArXiv.

[28]  David Zhang,et al.  Time Series Classification Using Support Vector Machine with Gaussian Elastic Metric Kernel , 2010, 2010 20th International Conference on Pattern Recognition.

[29]  Tim Oates,et al.  GrammarViz 2.0: A Tool for Grammar-Based Pattern Discovery in Time Series , 2014, ECML/PKDD.

[30]  Maya R. Gupta,et al.  Similarity-based Classification: Concepts and Algorithms , 2009, J. Mach. Learn. Res..

[31]  Claus Bahlmann,et al.  Learning with Distance Substitution Kernels , 2004, DAGM-Symposium.

[32]  Akira Hayashi,et al.  Embedding of time series data by using dynamic time warping distances , 2006, Systems and Computers in Japan.

[33]  Shigeki Sagayama,et al.  Dynamic Time-Alignment Kernel in Support Vector Machine , 2001, NIPS.

[34]  Chih-Jen Lin,et al.  A study on SMO-type decomposition methods for support vector machines , 2006, IEEE Trans. Neural Networks.

[35]  Sule Gündüz Ögüdücü,et al.  SAGA: A novel signal alignment method based on genetic algorithm , 2013, Inf. Sci..

[36]  Alexander J. Smola,et al.  Learning with non-positive kernels , 2004, ICML.

[37]  Zhi-Quan Luo,et al.  Guaranteed Matrix Completion via Nonconvex Factorization , 2014, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[38]  Jason Lines,et al.  Classification of time series by shapelet transformation , 2013, Data Mining and Knowledge Discovery.

[39]  Zheng Zhang,et al.  An Analysis of Transformation on Non - Positive Semidefinite Similarity Matrix for Kernel Machines , 2005, ICML 2005.

[40]  Klaus Obermayer,et al.  Support Vector Machines for Dyadic Data , 2006, Neural Computation.

[41]  Jinfeng Yi,et al.  Similarity Preserving Representation Learning for Time Series Analysis , 2017, ArXiv.

[42]  Jason Lines,et al.  Evaluating Improvements to the Shapelet Transform , 2016 .

[43]  Klaus Obermayer,et al.  Classification on Pairwise Proximity Data , 1998, NIPS.

[44]  Claus Bahlmann,et al.  Online handwriting recognition with support vector machines - a kernel approach , 2002, Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition.

[45]  Derong Liu TNNLS Call for Reviewers and Special Issues , 2015, IEEE Trans. Neural Networks Learn. Syst..

[46]  Eamonn J. Keogh,et al.  Experimental comparison of representation methods and distance measures for time series data , 2012, Data Mining and Knowledge Discovery.

[47]  Alicia Troncoso Lora,et al.  A multi-scale smoothing kernel for measuring time-series similarity , 2015, Neurocomputing.

[48]  Rohit J. Kate Using dynamic time warping distances as features for improved time series classification , 2016, Data Mining and Knowledge Discovery.

[49]  Tim Oates,et al.  RPM: Representative Pattern Mining for Efficient Time Series Classification , 2016, EDBT.

[50]  Mehryar Mohri,et al.  Rational Kernels: Theory and Algorithms , 2004, J. Mach. Learn. Res..

[51]  Padhraic Smyth,et al.  Clustering Sequences with Hidden Markov Models , 1996, NIPS.

[52]  Aristides Gionis,et al.  European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2014, Nancy, France, September 15-19, 2014 , 2014 .

[53]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[54]  Pierre-François Marteau,et al.  Discrete Elastic Inner Vector Spaces with Application to Time Series and Sequence Mining , 2013, IEEE Transactions on Knowledge and Data Engineering.

[55]  Azriel Rosenfeld,et al.  Machine Learning and Data Mining in Pattern Recognition , 2000, Lecture Notes in Computer Science.

[56]  Bing-Yu Sun,et al.  A Study on the Dynamic Time Warping in Kernel Machines , 2007, 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System.

[57]  Francisco Casacuberta,et al.  On the metric properties of dynamic time warping , 1987, IEEE Trans. Acoust. Speech Signal Process..

[58]  Laurent Amsaleg,et al.  Learning DTW-Preserving Shapelets , 2017, IDA.

[59]  Vipin Kumar,et al.  Introduction to Data Mining , 2022, Data Mining and Machine Learning Applications.

[60]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[61]  Jian Pei,et al.  A brief survey on sequence classification , 2010, SKDD.

[62]  JeongYoung-Seon,et al.  Support vector-based algorithms with weighted dynamic time warping kernel function for time series classification , 2015 .

[63]  Renée J. Miller,et al.  Similarity search over time-series data using wavelets , 2002, Proceedings 18th International Conference on Data Engineering.

[64]  Chiranjib Bhattacharyya,et al.  A large margin approach for writer independent online handwriting classification , 2008, Pattern Recognit. Lett..

[65]  Stefan Rüping,et al.  SVM Kernels for Time Series Analysis , 2001 .

[66]  Qinghua Hu,et al.  Kernel sparse representation for time series classification , 2015, Inf. Sci..

[67]  Fionn Murtagh,et al.  Ward’s Hierarchical Agglomerative Clustering Method: Which Algorithms Implement Ward’s Criterion? , 2011, Journal of Classification.

[68]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[69]  Edward Y. Chang,et al.  Learning with non-metric proximity matrices , 2005, MULTIMEDIA '05.

[70]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[71]  Chiranjib Bhattacharyya,et al.  Time Series Classification for Online Tamil Handwritten Character Recognition - A Kernel Based Approach , 2004, ICONIP.

[72]  Eamonn J. Keogh,et al.  Exact indexing of dynamic time warping , 2002, Knowledge and Information Systems.

[73]  Eamonn J. Keogh,et al.  DTW-D: time series semi-supervised learning from a single example , 2013, KDD.

[74]  Pierre-François Marteau,et al.  Time Warp Edit Distance with Stiffness Adjustment for Time Series Matching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[75]  Jason Lines,et al.  A shapelet transform for time series classification , 2012, KDD.

[76]  Jinfeng Yi,et al.  Random Warping Series: A Random Features Method for Time-Series Embedding , 2018, AISTATS.

[77]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[78]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[79]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[80]  Edwin R. Hancock,et al.  Spherical and Hyperbolic Embeddings of Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Raja Jayaraman,et al.  Support vector-based algorithms with weighted dynamic time warping kernel function for time series classification , 2015, Knowl. Based Syst..

[82]  Robert P. W. Duin,et al.  Prototype selection for dissimilarity-based classifiers , 2006, Pattern Recognit..

[83]  Cordelia Schmid,et al.  A time series kernel for action recognition , 2011, BMVC.

[84]  Marco Cuturi,et al.  Fast Global Alignment Kernels , 2011, ICML.

[85]  Hüseyin Kaya,et al.  A distance based time series classification framework , 2015, Inf. Syst..

[86]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[87]  Thomas Philip Runarsson,et al.  Support vector machines and dynamic time warping for time series , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[88]  Jason Weston,et al.  Dealing with large diagonals in kernel matrices , 2003 .

[89]  Pavlos Protopapas,et al.  Kernels for Periodic Time Series Arising in Astronomy , 2009, ECML/PKDD.

[90]  Tomoko Matsui,et al.  A Kernel for Time Series Based on Global Alignments , 2006, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[91]  Eamonn J. Keogh,et al.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances , 2016, Data Mining and Knowledge Discovery.

[92]  KeoghEamonn,et al.  On the Need for Time Series Data Mining Benchmarks , 2003 .

[93]  Gustavo E. A. P. A. Batista,et al.  Improved Time Series Classification with Representation Diversity and SVM , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[94]  Bernhard Schölkopf,et al.  Training Invariant Support Vector Machines , 2002, Machine Learning.

[95]  Akira Hayashi,et al.  Embedding Time Series Data for Classification , 2005, MLDM.

[96]  Li Wei,et al.  Fast time series classification using numerosity reduction , 2006, ICML.

[97]  Chandan Srivastava,et al.  Support Vector Data Description , 2011 .

[98]  Akira Hayashi,et al.  Embedding of time series data by using dynamic time warping distances , 2006 .

[99]  Fuzhen Zhuang,et al.  Fast Time Series Classification Based on Infrequent Shapelets , 2012, 2012 11th International Conference on Machine Learning and Applications.

[100]  Eamonn Keogh Exact Indexing of Dynamic Time Warping , 2002, VLDB.

[101]  Bin Ma,et al.  The similarity metric , 2001, IEEE Transactions on Information Theory.

[102]  Jinfeng Yi,et al.  Similarity Preserving Representation Learning for Time Series Clustering , 2019, IJCAI.

[103]  Jessica Lin,et al.  Evolving Separating References for Time Series Classification , 2018, SDM.

[104]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[105]  Anthony J. Bagnall,et al.  Binary Shapelet Transform for Multiclass Time Series Classification , 2015, Trans. Large Scale Data Knowl. Centered Syst..

[106]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[107]  Eamonn J. Keogh,et al.  Time series shapelets: a novel technique that allows accurate, interpretable and fast classification , 2010, Data Mining and Knowledge Discovery.

[108]  Robert P. W. Duin,et al.  Support Vector Data Description , 2004, Machine Learning.

[109]  Eamonn J. Keogh,et al.  Time series shapelets: a new primitive for data mining , 2009, KDD.

[110]  Marcella Corduas,et al.  Time series clustering and classification by the autoregressive metric , 2008, Comput. Stat. Data Anal..

[111]  Gareth J. Janacek,et al.  A Run Length Transformation for Discriminating Between Auto Regressive Time Series , 2014, J. Classif..

[112]  Joan Serrà,et al.  An empirical evaluation of similarity measures for time series classification , 2014, Knowl. Based Syst..

[113]  Daniel T. Larose,et al.  An Introduction to Data Mining , 2005 .

[114]  Alexander J. Smola,et al.  Learning with Kernels: support vector machines, regularization, optimization, and beyond , 2001, Adaptive computation and machine learning series.

[115]  Stephan Spiegel,et al.  Dimension Reduction in Dissimilarity Spaces for Time Series Classification , 2015, AALTD@PKDD/ECML.

[116]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2006 .

[117]  W. Greub Linear Algebra , 1981 .

[118]  Li Zhang,et al.  An Altered Kernel Transformation for Time Series Classification , 2017, ICONIP.

[119]  C. Adams Tales of Topology. (Book Reviews: The Knot Book. An Elementary Introduction to the Mathematical Theory of Knots.) , 1994 .

[120]  Dong Zhou,et al.  Translation techniques in cross-language information retrieval , 2012, CSUR.

[121]  Lei Chen,et al.  On The Marriage of Lp-norms and Edit Distance , 2004, VLDB.

[122]  Stephan K. Chalup,et al.  GDTW-P-SVMs: Variable-length time series analysis using support vector machines , 2013, Neurocomputing.

[123]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[124]  Sylvie Gibet,et al.  Constructing Positive Elastic Kernels with Application to Time Series Classification , 2010, ArXiv.

[125]  Deniz Erdogmus,et al.  A reproducing kernel Hilbert space framework for pairwise time series distances , 2008, ICML '08.