Hierarchical Bayesian Optimization Algorithm

The hierarchical Bayesian optimization algorithm (hBOA) solves nearly decomposable and hierarchical optimization problems scalably by combining concepts from evolutionary computation, machine learning and statistics. Since many complex real-world systems are nearly decomposable and hierarchical, hBOA is expected to provide scalable solutions for many complex real-world problems. This chapter describes hBOA and its predecessor, the Bayesian optimization algorithm (BOA), and outlines some of the most important theoretical and empirical results in this line of research.

[1]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[2]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[3]  D. Goldberg,et al.  Sporadic Model Building for Efficiency Enhancement of hBOA Martin Pelikan , 2005 .

[4]  Heinz Mühlenbein,et al.  How Genetic Algorithms Really Work: Mutation and Hillclimbing , 1992, PPSN.

[5]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[6]  Martin Pelikan,et al.  Fitness Inheritance in the Bayesian Optimization Algorithm , 2004, GECCO.

[7]  Nir Friedman,et al.  On the Sample Complexity of Learning Bayesian Networks , 1996, UAI.

[8]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[9]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[10]  D. Goldberg,et al.  Domino convergence, drift, and the temporal-salience structure of problems , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[11]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[12]  Shumeet Baluja,et al.  Using a priori knowledge to create probabilistic models for optimization , 2002, Int. J. Approx. Reason..

[13]  Martin Pelikan,et al.  Parameter-Less Hierarchical BOA , 2004, GECCO.

[14]  David Maxwell Chickering,et al.  Learning Bayesian networks: The combination of knowledge and statistical data , 1995, Mach. Learn..

[15]  Heinz Mühlenbein,et al.  On the Mean Convergence Time of Evolutionary Algorithms without Selection and Mutation , 1994, PPSN.

[16]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[17]  Kumara Sastry,et al.  Efficient Atomic Cluster Optimization Using A Hybrid Extended Compact Genetic Algorithm With Seeded , 2001 .

[18]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[19]  Jiri Ocenasek,et al.  Estimation of Distribution Algorithm for Mixed Continuous-Discrete Optimization , 2002 .

[20]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[21]  David E. Goldberg,et al.  Getting the best of both worlds: Discrete and continuous genetic and evolutionary algorithms in concert , 2003, Inf. Sci..

[22]  Martin V. Butz,et al.  Extracted global structure makes local building block processing effective in XCS , 2005, GECCO '05.

[23]  Martin Pelikan,et al.  Design of Multithreaded Estimation of Distribution Algorithms , 2003, GECCO.

[24]  Jordan B. Pollack,et al.  Modeling Building-Block Interdependency , 1998, PPSN.

[25]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[26]  David E. Goldberg,et al.  Time Complexity of genetic algorithms on exponentially scaled problems , 2000, GECCO.

[27]  Josef Schwarz,et al.  Estimation Distribution Algorithm for mixed continuous-discrete optimization problems , 2002 .

[28]  Thomas D. LaToza,et al.  On the supply of building blocks , 2001 .

[29]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[30]  Tian-Li Yu,et al.  Optimization of a Constrained Feed Network for an Antenna Array Using Simple and Competent Genetic Algorithm Techniques , 2004 .

[31]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[32]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[33]  David E. Goldberg,et al.  A hierarchy machine: Learning to optimize from nature and humans , 2003, Complex..

[34]  Marco Laumanns,et al.  Bayesian Optimization Algorithms for Multi-objective Optimization , 2002, PPSN.

[35]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[36]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[37]  David E. Goldberg,et al.  Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[38]  Schloss Birlinghoven,et al.  How Genetic Algorithms Really Work I.mutation and Hillclimbing , 2022 .

[39]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[40]  David E. Goldberg,et al.  Bayesian Optimization Algorithm, Population Sizing, and Time to Convergence , 2000, GECCO.

[41]  Martin Pelikan,et al.  Computational Complexity and Simulation of Rare Events of Ising Spin Glasses , 2004, GECCO.

[42]  Fernando G. Lobo,et al.  A parameter-less genetic algorithm , 1999, GECCO.

[43]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[44]  Dirk Thierens,et al.  Convergence Models of Genetic Algorithm Selection Schemes , 1994, PPSN.

[45]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[46]  Josef Schwarz,et al.  The Parallel Bayesian Optimization Algorithm , 2000 .

[47]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[48]  Franz Rothlauf,et al.  Towards a theory of representations for genetic and evolutionary algorithms - development of basic concepts and their application to binary and tree representations , 2001, Ausgezeichnete Informatikdissertationen.

[49]  Jiri Ocenasek,et al.  EXPERIMENTAL STUDY: HYPERGRAPH PARTITIONING BASED ON THE SIMPLE AND ADVANCED GENETIC ALGORITHM BMDA AND BOA , 2002 .

[50]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[51]  David E. Goldberg,et al.  Hierarchical BOA Solves Ising Spin Glasses and MAXSAT , 2003, GECCO.

[52]  Heinz Mühlenbein,et al.  Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning , 2002, Int. J. Approx. Reason..

[53]  Jingpeng Li,et al.  A Bayesian optimization algorithm for the nurse scheduling problem , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[54]  Kalyanmoy Deb,et al.  Sufficient conditions for deceptive and easy binary functions , 1994, Annals of Mathematics and Artificial Intelligence.

[55]  Ben Goertzel,et al.  Learning computer programs with the bayesian optimization algorithm , 2005, GECCO '05.

[56]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[57]  Franz Rothlauf,et al.  Bad Codings and the Utility of Well-Designed Genetic Algorithms , 2000, GECCO.

[58]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[59]  David E. Goldberg,et al.  Bayesian optimization algorithm: from single level to hierarchy , 2002 .

[60]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[61]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[62]  David E. Goldberg,et al.  Scalability of the Bayesian optimization algorithm , 2002, Int. J. Approx. Reason..

[63]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[64]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[65]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[66]  David E. Goldberg,et al.  Multiobjective hBOA, clustering, and scalability , 2005, GECCO '05.