What are dynamic optimization problems?

Dynamic Optimization Problems (DOPs) have been widely studied using Evolutionary Algorithms (EAs). Yet, a clear and rigorous definition of DOPs is lacking in the Evolutionary Dynamic Optimization (EDO) community. In this paper, we propose a unified definition of DOPs based on the idea of multiple-decision-making discussed in the Reinforcement Learning (RL) community. We draw a connection between EDO and RL by arguing that both of them are studying DOPs according to our definition of DOPs. We point out that existing EDO or RL research has been mainly focused on some types of DOPs. A conceptualized benchmark problem, which is aimed at the systematic study of various DOPs, is then developed. Some interesting experimental studies on the benchmark reveal that EDO and RL methods are specialized in certain types of DOPs and more importantly new algorithms for DOPs can be developed by combining the strength of both EDO and RL methods.

[1]  Hartmut Pohlheim,et al.  Optimal control of greenhouse climate using real-world weather data and evolutionary algorithms , 1999 .

[2]  Zbigniew Michalewicz,et al.  Adaptive Business Intelligence: Three Case Studies , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[3]  Xin Yao,et al.  Population-Based Incremental Learning With Associative Memory for Dynamic Environments , 2008, IEEE Transactions on Evolutionary Computation.

[4]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[5]  Susana Cecilia Esquivel,et al.  An Evolutionary Algorithm to Track Changes of Optimum Value Locations in Dynamic Environments , 2004 .

[6]  Hui Cheng,et al.  Genetic Algorithms With Immigrants and Memory Schemes for Dynamic Shortest Path Routing Problems in Mobile Ad Hoc Networks , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[8]  Karsten Weicker,et al.  An Analysis of Dynamic Severity and Population Size , 2000, PPSN.

[9]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Xin Yao,et al.  Dynamic combinatorial optimisation problems: an analysis of the subset sum problem , 2011, Soft Comput..

[12]  Jürgen Branke,et al.  Memory enhanced evolutionary algorithms for changing optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[13]  Zbigniew Michalewicz,et al.  Adaptation in Dynamic Environments: A Case Study in Mission Planning , 2012, IEEE Transactions on Evolutionary Computation.

[14]  Changhe Li,et al.  A Generalized Approach to Construct Benchmark Problems for Dynamic Optimization , 2008, SEAL.

[15]  Peter A. N. Bosman.  Learning and Anticipation in Online Dynamic Optimization , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[16]  Stephen F. Smith,et al.  A Memory Enhanced Evolutionary Algorithm for Dynamic Scheduling Problems , 2008, EvoWorkshops.

[17]  David E. Goldberg,et al.  Nonstationary Function Optimization Using Genetic Algorithms with Dominance and Diploidy , 1987, ICGA.

[18]  R.W. Morrison,et al.  A test problem generator for non-stationary environments , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[19]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[20]  Nate Kohl,et al.  Reinforcement Learning Benchmarks and Bake-offs II A workshop at the 2005 NIPS conference , 2005 .

[21]  Trung Thanh Nguyen,et al.  Continuous dynamic optimisation using evolutionary algorithms , 2011 .

[22]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[23]  Zbigniew Michalewicz,et al.  Searching for optima in non-stationary environments , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[24]  Zbigniew Michalewicz,et al.  Analysis and modeling of control tasks in dynamic systems , 2002, IEEE Trans. Evol. Comput..

[25]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[26]  Shengxiang Yang,et al.  Evolutionary dynamic optimization: A survey of the state of the art , 2012, Swarm Evol. Comput..

[27]  Xin Yao,et al.  Dual population-based incremental learning for problem optimization in dynamic environments , 2003 .