Evolutionary Multi-objective Ensemble Learning for Multivariate Electricity Consumption Prediction

Energy consumption prediction typically corresponds to a multivariate time series prediction task where different channels in the multivariate time series represent energy consumption data and various auxiliary data related to energy consumption such as environmental factors. It is non-trivial to resolve this task, which requires finding the most appropriate prediction model and the most useful features (extracted from the raw data) to be used by the model. This work proposes an evolutionary multi-objective ensemble learning (EMOEL) technique which uses extreme learning machines (ELMs) as base predictors due to its highly recognized efficacy. EMOEL employs evolutionary multi-objective optimization to search for the optimal parameters of the model as well as the optimal features fed into the model subjected to two conflicting criteria, i.e., accuracy and diversity. It leads to a Pareto front composed of non-dominated optimal solutions where each solution depicts the number of hidden neurons in the ELM, the selected channels in the multivariate time series, the selected feature extraction methods and the selected time windows applied to the selected channels. The optimal solutions in the Pareto front stand for different end-to-end prediction models which may lead to different prediction results. To boost ultimate prediction accuracy, the models with respect to these optimal solutions are linearly combined with combination coefficients being optimized via an evolutionary algorithm. We evaluate the proposed method in comparison to some existing prediction techniques on an Australian University based dataset, which demonstrates the superiority of the proposed method.

[1]  Guohai Liu,et al.  Building's electricity consumption prediction using optimized artificial neural networks and principal component analysis , 2015 .

[2]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[3]  Wei Sun,et al.  Forecasting Monthly Electric Energy Consumption Using Feature Extraction , 2011 .

[4]  Flora D. Salim,et al.  Multivariate electricity consumption prediction with Extreme Learning Machine , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[5]  Xin Yao,et al.  Simultaneous training of negatively correlated neural networks in an ensemble , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[7]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[8]  Xiaodong Li,et al.  Differential evolution on the CEC-2013 single-objective continuous optimization testbed , 2013, 2013 IEEE Congress on Evolutionary Computation.

[9]  Huanhuan Chen,et al.  Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[10]  Xiaodong Li,et al.  Effects of population initialization on differential evolution for large scale optimization , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[11]  Jing J. Liang,et al.  Performance Evaluation of Multiagent Genetic Algorithm , 2006, Natural Computing.

[12]  Zhongyi Hu,et al.  Comprehensive learning particle swarm optimization based memetic algorithm for model selection in short-term load forecasting using support vector regression , 2014, Appl. Soft Comput..

[13]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[14]  Jarmo Partanen,et al.  Forecasting electricity price and demand using a hybrid approach based on wavelet transform, ARIMA and neural networks , 2014 .

[15]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[16]  Flora D. Salim,et al.  Multi-resolution Selective Ensemble Extreme Learning Machine for Electricity Consumption Prediction , 2017, ICONIP.

[17]  Lynne E. Parker,et al.  Predicting future hourly residential electrical consumption: A machine learning case study , 2012 .

[18]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[19]  Kay Chen Tan,et al.  Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Xiaodong Li,et al.  Initialization methods for large scale global optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[21]  Pedro Antonio Gutiérrez,et al.  Negative Correlation Ensemble Learning for Ordinal Regression , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[23]  A. Arabali,et al.  A hybrid short-term load forecasting with a new input selection framework , 2015 .

[24]  Luiz Eduardo Soares de Oliveira,et al.  Feature selection for ensembles:a hierarchical multi-objective genetic algorithm approach , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[25]  Yaochu Jin,et al.  Evolutionary multi-objective generation of recurrent neural network ensembles for time series prediction , 2014, Neurocomputing.

[26]  Xin Yao,et al.  Ensemble Learning Using Multi-Objective Evolutionary Algorithms , 2006, J. Math. Model. Algorithms.

[27]  Yadong Mu,et al.  Piecewise linear approximation of streaming time series data with max-error guarantees , 2015, 2015 IEEE 31st International Conference on Data Engineering.

[28]  A. Kai Qin,et al.  An improved CUDA-based implementation of differential evolution on GPU , 2012, GECCO '12.

[29]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[30]  Xin Yao,et al.  Making use of population information in evolutionary artificial neural networks , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[31]  K. S. Banerjee Generalized Inverse of Matrices and Its Applications , 1973 .

[32]  C. R. Rao,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[33]  Shengwei Wang,et al.  Development of prediction models for next-day building energy consumption and peak power demand using data mining techniques , 2014 .

[34]  Song Li,et al.  An ensemble approach for short-term load forecasting by extreme learning machine , 2016 .

[35]  Guang-Bin Huang,et al.  Extreme Learning Machine for Multilayer Perceptron , 2016, IEEE Transactions on Neural Networks and Learning Systems.