Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms

Over the last few decades, genetic and evolutionary algorithms (GEAs) have been successfully applied to many problems of business, engineering, and science. This paper discusses probabilistic model-building genetic algorithms (PMBGAs), which are among the most important directions of current GEA research. PMBGAs replace traditional variation operators of GEAs by learning and sampling a probabilistic model of promising solutions. The paper describes two advanced PMBGAs: the Bayesian optimization algorithm (BOA), and the hierarchical BOA (hBOA). The paper argues that BOA and hBOA can solve an important class of nearly decomposable and hierarchical problems in a quadratic or subquadratic number of function evaluations with respect to the number of decision variables.

[1]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[2]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[3]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[4]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[5]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[6]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[7]  David E. Goldberg,et al.  Linkage Problem , Distribution Estimation , and Bayesian , 1998 .

[8]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[9]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[10]  David E. Goldberg,et al.  Scalability of the Bayesian optimization algorithm , 2002, Int. J. Approx. Reason..

[11]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[12]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[13]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[14]  David E. Goldberg,et al.  Bayesian Optimization Algorithm, Population Sizing, and Time to Convergence , 2000, GECCO.

[15]  Heinz Mühlenbein,et al.  How Genetic Algorithms Really Work: Mutation and Hillclimbing , 1992, PPSN.

[16]  Dirk Thierens,et al.  Mixing in Genetic Algorithms , 1993, ICGA.

[17]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[18]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[19]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[20]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[21]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[22]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[23]  David E. Goldberg,et al.  Bayesian Optimization Algorithm: From Single Level to Hierarchy , 2002 .

[24]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .