Multiobjective Learning in the Model Space for Time Series Classification

A well-defined distance is critical for the performance of time series classification. Existing distance measurements can be categorized into two branches. One is to utilize handmade features for calculating distance, e.g., dynamic time warping, which is limited to exploiting the dynamic information of time series. The other methods make use of the dynamic information by approximating the time series with a generative model, e.g., Fisher kernel. However, previous distance measurements for time series seldom exploit the label information, which is helpful for classification by distance metric learning. In order to attain the benefits of the dynamic information of time series and the label information simultaneously, this paper proposes a multiobjective learning algorithm for both time series approximation and classification, termed multiobjective model-metric (MOMM) learning. In MOMM, a recurrent network is exploited as the temporal filter, based on which, a generative model is learned for each time series as a representation of that series. The models span a non-Euclidean space, where the label information is utilized to learn the distance metric. The distance between time series is then calculated as the model distance weighted by the learned metric. The network size is also optimized to learn parsimonious representations. MOMM simultaneously optimizes the data representation, the time series model separation, and the network size. The experiments show that MOMM achieves not only superior overall performance on uni/multivariate time series classification but also promising time series prediction performance.

[1]  Yen-Ting Yeh,et al.  Multiobjective Evolution of Biped Robot Gaits Using Advanced Continuous Ant-Colony Optimized Recurrent Neural Networks , 2018, IEEE Transactions on Cybernetics.

[2]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[3]  Xin Yao,et al.  Ensemble Learning Using Multi-Objective Evolutionary Algorithms , 2006, J. Math. Model. Algorithms.

[4]  Dah-Jye Lee,et al.  Contour Matching for Fish Species Recognition and Migration Monitoring , 2008, Applications of Computational Intelligence in Biology.

[5]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2006 .

[6]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[7]  Xin Yao,et al.  Corner Sort for Pareto-Based Many-Objective Optimization , 2014, IEEE Transactions on Cybernetics.

[8]  Carlos Agón,et al.  Multiobjective Time Series Matching for Audio Classification and Retrieval , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[9]  O. J. Dunn Multiple Comparisons among Means , 1961 .

[10]  Eun-Soo Kim,et al.  Preference-Based Solution Selection Algorithm for Evolutionary Multiobjective Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[11]  Kay Chen Tan,et al.  Hybrid Multiobjective Evolutionary Design for Artificial Neural Networks , 2008, IEEE Transactions on Neural Networks.

[12]  Nuno Vasconcelos,et al.  A Kullback-Leibler Divergence Based Kernel for SVM Classification in Multimedia Applications , 2003, NIPS.

[13]  Tony Jebara,et al.  Probability Product Kernels , 2004, J. Mach. Learn. Res..

[14]  Tomoko Matsui,et al.  A Kernel for Time Series Based on Global Alignments , 2006, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[15]  Laurens van der Maaten,et al.  Learning Discriminative Fisher Kernels , 2011, ICML.

[16]  Li Wei,et al.  Experiencing SAX: a novel symbolic representation of time series , 2007, Data Mining and Knowledge Discovery.

[17]  Gary G. Yen,et al.  Minimum Manhattan Distance Approach to Multiple Criteria Decision Making in Multiobjective Optimization Problems , 2016, IEEE Transactions on Evolutionary Computation.

[18]  X. Yao,et al.  Model-based kernel for efficient time series analysis , 2013, KDD.

[19]  Peter Tiño,et al.  Simple Deterministically Constructed Cycle Reservoirs with Regular Jumps , 2012, Neural Computation.

[20]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[21]  Cory Merkel,et al.  Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing , 2016, Front. Neurosci..

[22]  Eamonn J. Keogh,et al.  Derivative Dynamic Time Warping , 2001, SDM.

[23]  Eamonn J. Keogh,et al.  Searching and Mining Trillions of Time Series Subsequences under Dynamic Time Warping , 2012, KDD.

[24]  Huanhuan Chen,et al.  Model Metric Co-Learning for Time Series Classification , 2015, IJCAI.

[25]  André Frank Krause,et al.  Multiobjective optimization of echo state networks for multiple motor pattern learning , 2010 .

[26]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[27]  J. Krug,et al.  Empirical fitness landscapes and the predictability of evolution , 2014, Nature Reviews Genetics.

[28]  Huanhuan Chen,et al.  Learning in the Model Space for Cognitive Fault Diagnosis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Kalyanmoy Deb,et al.  Finding Knees in Multi-objective Optimization , 2004, PPSN.

[30]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[31]  Hui Ding,et al.  Querying and mining of time series data: experimental comparison of representations and distance measures , 2008, Proc. VLDB Endow..

[32]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[33]  George C. Runger,et al.  A Bag-of-Features Framework to Classify Time Series , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Jenna Wiens,et al.  Patient Risk Stratification for Hospital-Associated C. diff as a Time-Series Classification Task , 2012, NIPS.

[35]  Manuel P. Cuéllar,et al.  Multiobjective Hybrid Optimization and Training of Recurrent Neural Networks , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[36]  Daoqiang Zhang,et al.  A Multiobjective Simultaneous Learning Framework for Clustering and Classification , 2010, IEEE Transactions on Neural Networks.

[37]  Jonathan Tompson,et al.  Unsupervised Learning of Spatiotemporally Coherent Metrics , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  David Haussler,et al.  Using the Fisher Kernel Method to Detect Remote Protein Homologies , 1999, ISMB.