Reinforcement learning for the unit commitment problem

In this work we solve the day-ahead unit commitment (UC) problem, by formulating it as a Markov decision process (MDP) and finding a low-cost policy for generation scheduling. We present two reinforcement learning algorithms, and devise a third one. We compare our results to previous work that uses simulated annealing (SA), and show a 27% improvement in operation costs, with running time of 2.5 minutes (compared to 2.5 hours of existing state-of-the-art).

[1]  F. N. Lee,et al.  Short-term thermal unit commitment-a new method , 1988 .

[2]  Grzegorz Dudek,et al.  Unit commitment by genetic algorithm with specialized search operators , 2004 .

[3]  Ruiwei Jiang,et al.  Robust Unit Commitment With Wind Power and Pumped Storage Hydro , 2012, IEEE Transactions on Power Systems.

[4]  A. Conejo,et al.  Optimal response of a thermal unit to an electricity spot market , 2000 .

[5]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[6]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT' 98.

[7]  B. F. Hobbs,et al.  Commitment and Dispatch With Uncertain Wind Generation by Dynamic Programming , 2012, IEEE Transactions on Sustainable Energy.

[8]  V. Palanisamy,et al.  A dynamic programming based fast computation Hopfield neural network for unit commitment and economic dispatch , 2007 .

[9]  Grzegorz Dudek,et al.  Adaptive simulated annealing schedule to the unit commitment problem , 2010 .

[10]  M. Anjos,et al.  Tight Mixed Integer Linear Programming Formulations for the Unit Commitment Problem , 2012, IEEE Transactions on Power Systems.

[11]  Tomonobu Senjyu,et al.  A technique for unit commitment with energy storage system , 2007 .

[12]  Gerald B. Sheblé,et al.  Unit commitment literature synopsis , 1994 .

[13]  Chuan-Ping Cheng,et al.  Unit commitment by Lagrangian relaxation and genetic algorithms , 2000 .

[14]  Walter L. Snyder,et al.  Dynamic Programming Approach to Unit Commitment , 1987, IEEE Transactions on Power Systems.

[15]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT.

[16]  Arthur I. Cohen,et al.  A Branch-and-Bound Algorithm for Unit Commitment , 1983, IEEE Transactions on Power Apparatus and Systems.

[17]  John N. Tsitsiklis,et al.  Neuro-dynamic programming: an overview , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[18]  N.P. Padhy,et al.  Unit commitment-a bibliographical survey , 2004, IEEE Transactions on Power Systems.

[19]  Malcolm Irving,et al.  Large scale unit commitment using a hybrid genetic algorithm , 1997 .

[20]  S. Virmani,et al.  Implementation of a Lagrangian Relaxation Based Unit Commitment Problem , 1989, IEEE Power Engineering Review.

[21]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[22]  M. Shahidehpour,et al.  Unit Commitment With Probabilistic Spinning Reserve and Interruptible Load Considerations , 2009, IEEE Transactions on Power Systems.

[23]  M. Carrion,et al.  A computationally efficient mixed-integer linear formulation for the thermal unit commitment problem , 2006, IEEE Transactions on Power Systems.

[24]  Tomonobu Senjyu,et al.  Fuzzy unit commitment solution—A novel twofold simulated annealing approach , 2007 .

[25]  Alberto Borghetti,et al.  Using of a cost-based unit commitment algorithm to assist bidding strategy decisions , 2003, 2003 IEEE Bologna Power Tech Conference Proceedings,.

[26]  S. M. Shahidehpour,et al.  Short-term unit commitment expert system , 1990 .