Deep learning for load forecasting with smart meter data: Online Adaptive Recurrent Neural Network

Abstract Electricity load forecasting has been attracting research and industry attention because of its importance for energy management, infrastructure planning, and budgeting. In recent years, the proliferation of smart meters and other sensors has created new opportunities for sensor-based load forecasting on the building and even individual household level. Machine learning approaches such as Recurrent Neural Networks (RNNs) have shown great successes in load forecasting, but these approaches employ offline learning: they are trained once and miss on the opportunity to learn from newly arriving data. Moreover, they are not well suited for handling the concept drift; for example, their predictive performance will degrade if the load changes due to the installation of new equipment. Consequently, this paper proposes Online Adaptive RNN, an approach for load forecasting capable of continuously learning from newly arriving data and adapting to new patterns. RNN is employed to capture time dependencies while the online aspect is achieved by updating the RNN weights according to new data. The performance is monitored; if it degrades, online tuning is activated to adapt the RNN hyperparameters to changes in data. The proposed approach was evaluated with data from five individual homes: the results show that the proposed approach achieves higher accuracy than the standalone offline long short term memory network and five other online algorithms. Moreover, the time to learn from new samples is only a fraction of the time needed to re-train the offline model.

[1]  P. Alam.  ‘G’ , 2021, Composites Engineering: An A–Z Guide.

[2]  Aaron Klein,et al.  Hyperparameter Optimization , 2017, Encyclopedia of Machine Learning and Data Mining.

[3]  Alexey Tsymbal,et al.  The problem of concept drift: definitions and related work , 2004 .

[4]  Mohamed A. Meguid,et al.  Robust ensemble learning framework for day-ahead forecasting of household based energy consumption , 2018 .

[5]  Stuart J. Russell,et al.  Online bagging and boosting , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[6]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[7]  Karl Aberer,et al.  Robust Online Time Series Prediction with Recurrent Neural Networks , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[8]  Krithi Ramamritham,et al.  A hybrid model for building energy consumption forecasting using long short term memory networks , 2020 .

[9]  Xiao Jing,et al.  Short-Term Electricity Load Forecasting Model Based on EMD-GRU with Feature Selection , 2019, Energies.

[10]  Miriam A. M. Capretz,et al.  Energy Consumption Prediction with Big Data: Balancing Prediction Accuracy and Computational Resources , 2016, 2016 IEEE International Congress on Big Data (BigData Congress).

[11]  Gilles Notton,et al.  Forecasting of Intermittent Solar Energy Resource , 2018 .

[12]  Qinghua Hu,et al.  Robust functional regression for wind speed forecasting based on Sparse Bayesian learning , 2019, Renewable Energy.

[13]  Michelangelo Ceci,et al.  Spatial autocorrelation and entropy for renewable energy forecasting , 2018, Data Mining and Knowledge Discovery.

[14]  Anima Anandkumar,et al.  Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods , 2017 .

[15]  Duc-Long Luong,et al.  Nature-inspired metaheuristic ensemble model for forecasting energy consumption in residential buildings , 2020 .

[16]  Ali Ouni,et al.  Optimal Deep Learning LSTM Model for Electric Load Forecasting using Feature Selection and Genetic Algorithm: Comparison with Machine Learning Approaches † , 2018, Energies.

[17]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[18]  Aaron C. Courville,et al.  Recurrent Batch Normalization , 2016, ICLR.

[19]  George Hsieh,et al.  Mitigating Drift in Time Series Data with Noise Augmentation , 2019, 2019 International Conference on Computational Science and Computational Intelligence (CSCI).

[20]  Katarina Grolinger,et al.  Deep Learning for Load Forecasting: Sequence to Sequence Recurrent Neural Networks With Attention , 2020, IEEE Access.

[21]  Yi Wang,et al.  Review of Smart Meter Data Analytics: Applications, Methodologies, and Challenges , 2018, IEEE Transactions on Smart Grid.

[22]  Peter Lund,et al.  Energy and climate change , 2018 .

[23]  Richa Negi,et al.  Wind Power Forecasting Using Hybrid ARIMA-ANN Technique , 2019, Advances in Intelligent Systems and Computing.

[24]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[25]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[26]  P. Alam.  ‘O’ , 2021, Composites Engineering: An A–Z Guide.

[27]  João Gama,et al.  On evaluating stream learning algorithms , 2012, Machine Learning.

[28]  Tong Zhang,et al.  Accelerating Stochastic Gradient Descent using Predictive Variance Reduction , 2013, NIPS.

[29]  Wenjie Gang,et al.  Assessment of deep recurrent neural network-based strategies for short-term building energy predictions , 2019, Applied Energy.

[30]  Robert Latham,et al.  Adaptive Learning for Concept Drift in Application Performance Modeling , 2019, ICPP.

[31]  Jianchun Peng,et al.  A review of deep learning for renewable energy forecasting , 2019, Energy Conversion and Management.

[32]  Guobin Xu,et al.  Towards Online Deep Learning-Based Energy Forecasting , 2019, 2019 28th International Conference on Computer Communication and Networks (ICCCN).

[33]  T. Funabashi,et al.  One-Hour-Ahead Load Forecasting Using Neural Networks , 2002 .

[34]  Tanveer Ahmad,et al.  A review on renewable energy and electricity requirement forecasting models for smart grid and buildings , 2020 .

[35]  Mykola Pechenizkiy,et al.  An Overview of Concept Drift Applications , 2016 .

[36]  Mohammad Navid Fekri,et al.  Generating Energy Data for Machine Learning with Recurrent Generative Adversarial Networks , 2019, Energies.

[37]  Francis Bach,et al.  SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives , 2014, NIPS.

[38]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[39]  Javier J. Sánchez Medina,et al.  Data Stream Mining Applied to Maximum Wind Forecasting in the Canary Islands , 2019, Sensors.

[40]  Ricard Gavaldà,et al.  Learning from Time-Changing Data with Adaptive Windowing , 2007, SDM.

[41]  Hans-Peter Kriegel,et al.  Spiral Recurrent Neural Network for Online Learning , 2007, ESANN.

[42]  João Gama,et al.  Issues in evaluation of stream learning algorithms , 2009, KDD.

[43]  Li Li,et al.  Mid-to-long term wind and photovoltaic power generation prediction based on copula function and long short term memory network , 2019, Applied Energy.

[44]  Sham M. Kakade,et al.  Competing with the Empirical Risk Minimizer in a Single Pass , 2014, COLT.

[45]  E. S. Page.  CONTINUOUS INSPECTION SCHEMES , 1954 .

[46]  Srikanta Tirthapura,et al.  Variance-Reduced Stochastic Gradient Descent on Streaming Data , 2018, NeurIPS.

[47]  Stefan Kramer,et al.  Integrating LSTMs with Online Density Estimation for the Probabilistic Forecast of Energy Consumption , 2019, DS.

[48]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.