Walk-forward empirical wavelet random vector functional link for time series forecasting

Abstract The challenge of accurately forecasting a time series covers numerous disciplines, from economics to engineering. Among the thousands of machine learning models, random vector functional link (RVFL) is a robust and efficient model which has demonstrated its success in various challenging forecasting problems. RVFL is an efficient universal function appropriator that randomly generates the weights between the input and hidden layers. However, RVFL still lacks the strong ability to extract meaningful multi-scale features from input data because of the single-layer random mapping of enhancement nodes. Therefore, we propose to combine the empirical wavelet transformation (EWT) with RVFL to strengthen the multi-scale feature extraction ability. The EWT can decompose the original time series into several sub-series which carry the information of different frequencies. Besides, we propose a walk-forward decomposition mechanism to implement the EWT. By introducing such a walk-forward mechanism and the combination of EWT and RVFL, the hybrid model achieves high accuracy and averts the data leakage problem during forecasting. A detailed and comprehensive empirical study on twenty-six public time series validates the proposed model’s superiority compared with ten popular baseline models from the literature.

[1]  Dominique Zosso,et al.  Variational Mode Decomposition , 2014, IEEE Transactions on Signal Processing.

[2]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[3]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[4]  Johan A. K. Suykens,et al.  Financial time series prediction using least squares support vector machines within the evidence framework , 2001, IEEE Trans. Neural Networks.

[5]  Amir F. Atiya,et al.  An Empirical Comparison of Machine Learning Models for Time Series Forecasting , 2010 .

[6]  Ponnuthurai N. Suganthan,et al.  Random vector functional link network for short-term electricity load demand forecasting , 2016, Inf. Sci..

[7]  Robert P. W. Duin,et al.  Feedforward neural networks with random weights , 1992, Proceedings., 11th IAPR International Conference on Pattern Recognition. Vol.II. Conference B: Pattern Recognition Methodology and Systems.

[8]  Leandro dos Santos Coelho,et al.  Enhanced ensemble structures using wavelet neural networks applied to short-term load forecasting , 2019, Eng. Appl. Artif. Intell..

[9]  José Manuel Benítez,et al.  On the use of cross-validation for time series predictor evaluation , 2012, Inf. Sci..

[10]  P. Casazza THE ART OF FRAME THEORY , 1999, math/9910168.

[11]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[12]  J. Contreras,et al.  ARIMA models to predict next-day electricity prices , 2002 .

[13]  Ponnuthurai N. Suganthan,et al.  Ensemble Incremental Random Vector Functional Link Network for Short-term Crude Oil Price Forecasting , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[14]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[15]  Ling Yang,et al.  DSTP-RNN: a dual-stage two-phase attention-based recurrent neural networks for long-term and multivariate time series prediction , 2019, Expert Syst. Appl..

[16]  Okan Duru,et al.  Modeling principles in fuzzy time series forecasting , 2012, 2012 IEEE Conference on Computational Intelligence for Financial Engineering & Economics (CIFEr).

[17]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[18]  Christoph Bergmeir,et al.  Forecasting across time series databases using recurrent neural networks on groups of similar series: A clustering approach , 2017, Expert Syst. Appl..

[19]  Dejan J. Sobajic,et al.  Neural-net computing and the intelligent control of systems , 1992 .

[20]  Jing Liu,et al.  Time-Series Forecasting Based on High-Order Fuzzy Cognitive Maps and Wavelet Transform , 2018, IEEE Transactions on Fuzzy Systems.

[21]  Saifur Rahman,et al.  A real-time short-term load forecasting system using functional link network , 1997 .

[22]  Bin Ran,et al.  Travel time prediction for highway network based on the ensemble empirical mode decomposition and random vector functional link network , 2018, Appl. Soft Comput..

[23]  Ponnuthurai N. Suganthan,et al.  Detecting Wind Power Ramp with Random Vector Functional Link (RVFL) Network , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[24]  Mark J. Shensa,et al.  The discrete wavelet transform: wedding the a trous and Mallat algorithms , 1992, IEEE Trans. Signal Process..

[25]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[26]  Jie Chen,et al.  Air compressor load forecasting using artificial neural network , 2020 .

[27]  J. Contreras,et al.  ARIMA Models to Predict Next-Day Electricity Prices , 2002, IEEE Power Engineering Review.

[28]  J. Pratt Remarks on Zeros and Ties in the Wilcoxon Signed Rank Procedures , 1959 .

[29]  P. N. Suganthan,et al.  A comprehensive evaluation of random vector functional link networks , 2016, Inf. Sci..

[30]  Jérôme Gilles,et al.  Empirical Wavelet Transform , 2013, IEEE Transactions on Signal Processing.

[31]  Alejandro Cervantes,et al.  Convolution on neural networks for high-frequency trend prediction of cryptocurrency exchange rates using technical indicators , 2020, Expert Syst. Appl..

[32]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[33]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[34]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[35]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[36]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[37]  Liu Peng Combined Model Based on EMD-SVM for Short-term Wind Power Prediction , 2011 .

[38]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[39]  Ponnuthurai N. Suganthan,et al.  Ensemble incremental learning Random Vector Functional Link network for short-term electric load forecasting , 2018, Knowl. Based Syst..

[40]  Sander Bohte,et al.  Conditional Time Series Forecasting with Convolutional Neural Networks , 2017, 1703.04691.

[41]  Ravi Sankar,et al.  Time Series Prediction Using Support Vector Machines: A Survey , 2009, IEEE Computational Intelligence Magazine.

[42]  Garrison W. Cottrell,et al.  A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction , 2017, IJCAI.

[43]  Okan Duru,et al.  A fuzzy integrated logical forecasting model for dry bulk shipping index forecasting: An improved fuzzy time series approach , 2010, Expert Syst. Appl..

[44]  Ponnuthurai Nagaratnam Suganthan,et al.  Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting , 2017, Appl. Soft Comput..

[45]  Ponnuthurai Nagaratnam Suganthan,et al.  Electricity load demand time series forecasting with Empirical Mode Decomposition based Random Vector Functional Link network , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[46]  Guoqiang Peter Zhang,et al.  Time series forecasting using a hybrid ARIMA and neural network model , 2003, Neurocomputing.

[47]  Liang Du,et al.  Robust empirical wavelet fuzzy cognitive map for time series forecasting , 2020, Eng. Appl. Artif. Intell..

[48]  Maysam Abbod,et al.  A new hybrid financial time series prediction model , 2020, Eng. Appl. Artif. Intell..

[49]  P. N. Suganthan,et al.  Benchmarking Ensemble Classifiers with Novel Co-Trained Kernal Ridge Regression and Random Vector Functional Link Ensembles [Research Frontier] , 2017, IEEE Computational Intelligence Magazine.

[50]  J. Spencer Ten lectures on the probabilistic method , 1987 .

[51]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[52]  Mohammad Khorsand Zak,et al.  A hybridized intelligence model to improve the predictability level of strength index parameters of rocks , 2020, Neural Comput. Appl..

[53]  Gabriel Rilling,et al.  Empirical mode decomposition as a filter bank , 2004, IEEE Signal Processing Letters.

[54]  Seyed Abbas Hosseini,et al.  Updating the neural network sediment load models using different sensitivity analysis methods: a regional application , 2020 .

[55]  Witold Pedrycz,et al.  Fuzzy cognitive maps in the modeling of granular time series , 2017, Knowl. Based Syst..

[56]  Ranjeeta Bisoi,et al.  Modes decomposition method in fusion with robust random vector functional link network for crude oil price forecasting , 2019, Appl. Soft Comput..

[57]  Dejan J. Sobajic,et al.  Learning and generalization characteristics of the random vector Functional-link net , 1994, Neurocomputing.

[58]  Le Zhang,et al.  Visual Tracking With Convolutional Random Vector Functional Link Network , 2017, IEEE Transactions on Cybernetics.