Ensemble deep learning for regression and time series forecasting

In this paper, for the first time, an ensemble of deep learning belief networks (DBN) is proposed for regression and time series forecasting. Another novel contribution is to aggregate the outputs from various DBNs by a support vector regression (SVR) model. We show the advantage of the proposed method on three electricity load demand datasets, one artificial time series dataset and three regression datasets over other benchmark methods.

[1]  Luis Neves,et al.  Short‐term load forecasting based on support vector regression and load profiling , 2014 .

[2]  John R. Williams,et al.  Towards Accurate Electricity Load Forecasting in Smart Grids , 2012, DBKDA 2012.

[3]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[4]  Snehamoy Chatterjee,et al.  Ensemble Support Vector Machine Algorithm for Reliability Estimation of a Mining Machine , 2015, Qual. Reliab. Eng. Int..

[5]  Lahouari Ghouti,et al.  Efficient prediction of software fault proneness modules using support vector machines and probabilistic neural networks , 2011, 2011 Malaysian Conference in Software Engineering.

[6]  Wei-Chiang Hong,et al.  Seasonal Support Vector Regression with Chaotic Genetic Algorithm in Electric Load Forecasting , 2012, 2012 Sixth International Conference on Genetic and Evolutionary Computing.

[7]  R. Pace,et al.  Sparse spatial autoregressions , 1997 .

[8]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[9]  Rasmus Berg Palm,et al.  Prediction as a candidate for learning deep hierarchical models of data , 2012 .

[10]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[11]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[12]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[13]  Leo Breiman,et al.  Bias, Variance , And Arcing Classifiers , 1996 .

[14]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[15]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[16]  F. Galton.  Kinship and Correlation , 1989 .

[17]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[18]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[19]  Chuang Zhang,et al.  Horizontal and Vertical Ensemble with Deep Representation for Classification , 2013, ArXiv.

[20]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[21]  Kunikazu Kobayashi,et al.  Time Series Forecasting Using Restricted Boltzmann Machine , 2012, ICIC.

[22]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[23]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[24]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[25]  Pierre Geurts,et al.  Extremely randomized trees , 2006, Machine Learning.

[26]  Ian Osband,et al.  Deep Learning for Time Series Modeling CS 229 Final Project Report , 2012 .

[27]  Enrique Romero,et al.  Comparing Support Vector Machines and Feedforward Neural Networks With Similar Hidden-Layer Weights , 2007, IEEE Transactions on Neural Networks.