Oblique Random Forests on Residual Network Features

Time series are usually complicated in nature and contains many complex patterns. As such, many researchers work on different ways to pick up such patterns. In this paper, we explore using Residual Networks (a Convolutional Neural Network) as a feature extractor for Oblique Random Forest. Here, we extract features using Residual Networks, and pass the extracted feature set to Oblique Random Forest for classification of time series. Based on the experiments on 85 UCR datasets, we found that using features extracted from Residual Network significantly improves the performance of Oblique Random Forest. In addition, using including intermediate features from Residual Networks significantly improves the performance of Oblique Random Forests.

[1]  Ying Wah Teh,et al.  Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges , 2018, Expert Syst. Appl..

[2]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[3]  George C. Runger,et al.  A time series forest for classification and feature extraction , 2013, Inf. Sci..

[4]  Matteo Terzi,et al.  Time-Series Classification Methods: Review and Applications to Power Systems Data , 2018 .

[5]  Lifeng Shen,et al.  Time series classification with Echo Memory Networks , 2019, Neural Networks.

[6]  Jason Lines,et al.  Time series classification with ensembles of elastic distance measures , 2015, Data Mining and Knowledge Discovery.

[7]  Jeffrey Dean,et al.  Scalable and accurate deep learning with electronic health records , 2018, npj Digital Medicine.

[8]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[9]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Tomasz Górecki,et al.  Using derivatives in time series classification , 2012, Data Mining and Knowledge Discovery.

[11]  Ponnuthurai N. Suganthan,et al.  Enhancing Multi-Class Classification of Random Forest using Random Vector Functional Neural Network and Oblique Decision Surfaces , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[12]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[13]  Patrick Schäfer.  The BOSS is concerned with time series classification in the presence of noise , 2014, Data Mining and Knowledge Discovery.

[14]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[15]  George C. Runger,et al.  A Bag-of-Features Framework to Classify Time Series , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[17]  Eamonn J. Keogh,et al.  The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances , 2016, Data Mining and Knowledge Discovery.

[18]  Ponnuthurai N. Suganthan,et al.  Oblique Decision Tree Ensemble via Multisurface Proximal Support Vector Machine , 2015, IEEE Transactions on Cybernetics.

[19]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[20]  Xindong Wu,et al.  10 Challenging Problems in Data Mining Research , 2006, Int. J. Inf. Technol. Decis. Mak..

[21]  Anthony J. Bagnall,et al.  Binary Shapelet Transform for Multiclass Time Series Classification , 2015, Trans. Large Scale Data Knowl. Centered Syst..

[22]  Tomasz Górecki,et al.  Non-isometric transforms in time series classification using DTW , 2014, Knowl. Based Syst..

[23]  Lars Schmidt-Thieme,et al.  Learning time-series shapelets , 2014, KDD.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Jason Lines,et al.  Classification of time series by shapelet transformation , 2013, Data Mining and Knowledge Discovery.