Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting

Graphical abstractDisplay Omitted HighlightsAn ensemble deep learning method has been proposed for load demand forecasting.The hybrid method composes of Empirical Mode Decomposition and Deep Belief Network.Empirical Mode Decomposition based methods outperform the single structure models.Deep learning shows more advantages when the forecasting horizon increases. Load demand forecasting is a critical process in the planning of electric utilities. An ensemble method composed of Empirical Mode Decomposition (EMD) algorithm and deep learning approach is presented in this work. For this purpose, the load demand series were first decomposed into several intrinsic mode functions (IMFs). Then a Deep Belief Network (DBN) including two restricted Boltzmann machines (RBMs) was used to model each of the extracted IMFs, so that the tendencies of these IMFs can be accurately predicted. Finally, the prediction results of all IMFs can be combined by either unbiased or weighted summation to obtain an aggregated output for load demand. The electricity load demand data sets from Australian Energy Market Operator (AEMO) are used to test the effectiveness of the proposed EMD-based DBN approach. Simulation results demonstrated attractiveness of the proposed method compared with nine forecasting methods.

[1]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[2]  Chao Chen,et al.  A hybrid model for wind speed prediction using empirical mode decomposition and artificial neural networks , 2012 .

[3]  Le Zhang,et al.  Ensemble deep learning for regression and time series forecasting , 2014, 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL).

[4]  Irena Koprinska,et al.  Combining pattern sequence similarity with neural networks for forecasting electricity demand time series , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[5]  Jianzhou Wang,et al.  A trend fixed on firstly and seasonal adjustment model combined with the ε-SVR for short-term forecasting of electricity demand , 2009 .

[6]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[7]  T. Hesterberg,et al.  A regression-based approach to short-term system load forecasting , 1989, Conference Papers Power Industry Computer Application Conference.

[8]  Luca Maria Gambardella,et al.  Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition , 2010, ArXiv.

[9]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[10]  Chia-Nan Ko,et al.  Short-term load forecasting using SVR (support vector regression)-based radial basis function neural network with dual extended Kalman filter , 2013 .

[11]  Girijesh Prasad,et al.  EWMA model based shift-detection methods for detecting covariate shifts in non-stationary environments , 2015, Pattern Recognit..

[12]  P. N. Suganthan,et al.  A Comparative Study of Empirical Mode Decomposition-Based Short-Term Wind Speed Forecasting Methods , 2015, IEEE Transactions on Sustainable Energy.

[13]  Wei-Chiang Hong,et al.  Chaotic particle swarm optimization algorithm in a support vector regression electric load forecasting model , 2009 .

[14]  Luis Neves,et al.  Short‐term load forecasting based on support vector regression and load profiling , 2014 .

[15]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[16]  Ponnuthurai N. Suganthan,et al.  Random vector functional link network for short-term electricity load demand forecasting , 2016, Inf. Sci..

[17]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[18]  Wei-Chiang Hong,et al.  Electric load forecasting by seasonal recurrent SVR (support vector regression) with chaotic artific , 2011 .

[19]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[20]  Gabriel Rilling,et al.  On empirical mode decomposition and its algorithms , 2003 .

[21]  Le Zhang,et al.  A survey of randomized algorithms for training neural networks , 2016, Inf. Sci..

[22]  Fionn Murtagh,et al.  Wavelet-based nonlinear multiscale decomposition model for electricity load forecasting , 2006, Neurocomputing.

[23]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[24]  Senén Barro,et al.  Do we need hundreds of classifiers to solve real world classification problems? , 2014, J. Mach. Learn. Res..

[25]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[26]  Hironobu Fujiyoshi,et al.  To Be Bernoulli or to Be Gaussian, for a Restricted Boltzmann Machine , 2014, 2014 22nd International Conference on Pattern Recognition.

[27]  Jaime Lloret,et al.  Artificial neural networks for short-term load forecasting in microgrids environment , 2014 .

[28]  Geoffrey E. Hinton A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[29]  Ian Osband,et al.  Deep Learning for Time Series Modeling CS 229 Final Project Report , 2012 .

[30]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Enrique Romero,et al.  Comparing Support Vector Machines and Feedforward Neural Networks With Similar Hidden-Layer Weights , 2007, IEEE Transactions on Neural Networks.

[32]  Francisco Martínez-Álvarez,et al.  A Survey on Data Mining Techniques Applied to Electricity-Related Time Series Forecasting , 2015 .

[33]  Per Ask,et al.  A pattern recognition framework for detecting dynamic changes on cyclic time series , 2015, Pattern Recognit..

[34]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[35]  Liu Peng Combined Model Based on EMD-SVM for Short-term Wind Power Prediction , 2011 .

[36]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[37]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[38]  C. Holt Author's retrospective on ‘Forecasting seasonals and trends by exponentially weighted moving averages’ , 2004 .

[39]  Liang-Ying Wei,et al.  A hybrid ANFIS model based on empirical mode decomposition for stock time series forecasting , 2016, Appl. Soft Comput..

[40]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[41]  Ponnuthurai N. Suganthan,et al.  Oblique Decision Tree Ensemble via Multisurface Proximal Support Vector Machine , 2015, IEEE Transactions on Cybernetics.

[42]  Mohammad Bagher Menhaj,et al.  A hybrid short-term load forecasting with a new data preprocessing framework , 2015 .

[43]  Hyun Ah Song,et al.  Hierarchical Representation Using NMF , 2013, ICONIP.

[44]  Zhongyi Hu,et al.  Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression , 2014, Appl. Soft Comput..

[45]  Jianzhou Wang,et al.  An adaptive fuzzy combination model based on self-organizing map and support vector regression for electric load forecasting , 2012 .

[46]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[47]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Kunikazu Kobayashi,et al.  Time Series Forecasting Using Restricted Boltzmann Machine , 2012, ICIC.

[49]  Shen Lincheng,et al.  A new method for mitigation of end effect in empirical mode decomposition , 2010, 2010 2nd International Asia Conference on Informatics in Control, Automation and Robotics (CAR 2010).

[50]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[51]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[52]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[53]  J. Koenderink Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[54]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[55]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[56]  Georges A. Darbellay,et al.  Forecasting the short-term demand for electricity: Do neural networks stand a better chance? , 2000 .

[57]  Rasmus Berg Palm,et al.  Prediction as a candidate for learning deep hierarchical models of data , 2012 .

[58]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[59]  Ponnuthurai N. Suganthan,et al.  Random Forests with ensemble of feature spaces , 2014, Pattern Recognit..

[60]  Ponnuthurai N. Suganthan,et al.  A Novel Empirical Mode Decomposition With Support Vector Regression for Wind Speed Forecasting , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[61]  Irena Koprinska,et al.  Correlation and instance based feature selection for electricity load forecasting , 2015, Knowl. Based Syst..

[62]  Li-Chih Ying,et al.  Using adaptive network based fuzzy inference system to forecast regional electricity loads , 2008 .

[63]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.