A hierarchy machine: Learning to optimize from nature and humans

This article proposes a competent hierarchical optimization method called the hierarchical Bayesian optimization algorithm (hBOA). hBOA extends the Bayesian optimization algorithm (BOA) by incorporating three important features for robust and scalable optimization of hierarchical problems: proper decomposition, chunking, and preservation of alternative solutions. Additionally, the article proposes a class of difficult hierarchical problems called hierarchical traps, hBOA is shown to provide a scalable solution to the class of hierarchically decomposable problems and anything easier. Specifically, hBOA can solve hierarchical traps and other nearly decomposable problems in approximately O(n1.55 log n) to O(n2) function evaluations, where n is the number of decision variables in the problem.

[1]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[2]  Mark Stefik,et al.  Planning and Meta-Planning (MOLGEN: Part 2) , 1981, Artif. Intell..

[3]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[4]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[5]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[8]  Martin Pelikan,et al.  Bayesian Optimization Algorithm , 2005 .

[9]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[10]  G. Harik Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[11]  David E. Goldberg,et al.  Bayesian optimization algorithm: from single level to hierarchy , 2002 .

[12]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[13]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[14]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[15]  Jordan B. Pollack,et al.  Modeling Building-Block Interdependency , 1998, PPSN.

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  Hillol Kargupta,et al.  Function induction, gene expression, and evolutionary representation construction , 1999 .

[18]  David E. Goldberg,et al.  Bayesian Optimization Algorithm, Population Sizing, and Time to Convergence , 2000, GECCO.

[19]  Kalyanmoy Deb,et al.  Sufficient conditions for deceptive and easy binary functions , 1994, Annals of Mathematics and Artificial Intelligence.

[20]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[21]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[22]  Mark Stefik,et al.  Planning with Constraints (MOLGEN: Part 1) , 1981, Artif. Intell..

[23]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[24]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[25]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[26]  Earl D. Sacerdoti,et al.  The Nonlinear Nature of Plans , 1975, IJCAI.

[27]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[28]  Wray L. Buntine Theory Refinement on Bayesian Networks , 1991, UAI.

[29]  Kumara Sastry,et al.  Efficient Atomic Cluster Optimization Using A Hybrid Extended Compact Genetic Algorithm With Seeded , 2001 .

[30]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.