Order or not: does parallelization of model building in hBOA affect its scalability?

It has been shown that model building in the hierarchical Bayesian optimization algorithm (hBOA) can be efficiently parallelized by randomly generating an ancestral ordering of the nodes of the network prior to learning the network structure and allowing only dependencies consistent with the generated ordering. However, it has not been thoroughly shown that this approach to restricting probabilistic models does not affect scalability of hBOA on important classes of problems. This paper demonstrates that although the use of a random ancestral ordering restricts the structure of considered models to allow efficient parallelization of model building, its effects on hBOA performance and scalability are negligible.

[1]  Martin Pelikan,et al.  Design of Parallel Estimation of Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[3]  David E. Goldberg,et al.  Efficiency Enhancement of Estimation of Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[4]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[5]  David H. Ackley,et al.  An empirical study of bit vector function optimization , 1987 .

[6]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[7]  A. Hartmann.  Ground-state clusters of two-, three-, and four-dimensional +/-J Ising spin glasses. , 1999, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[9]  Martin Pelikan,et al.  Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications (Studies in Computational Intelligence) , 2006 .

[10]  Alexander Mendiburu,et al.  Parallel Estimation of Distribution Algorithms: New Approaches , 2003 .

[11]  David E. Goldberg,et al.  Hierarchical Bayesian Optimization Algorithm , 2006, Scalable Optimization via Probabilistic Modeling.

[12]  Martin Pelikan,et al.  Design of Multithreaded Estimation of Distribution Algorithms , 2003, GECCO.

[13]  David E. Goldberg,et al.  Bayesian Optimization Algorithm: From Single Level to Hierarchy , 2002 .

[14]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[15]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[16]  Franz Rothlauf,et al.  Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms , 2004 .

[17]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[18]  David E. Goldberg,et al.  Sporadic model building for efficiency enhancement of hierarchical BOA , 2006, GECCO.

[19]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[20]  Martin Pelikan,et al.  Searching for Ground States of Ising Spin Glasses with Hierarchical BOA and Cluster Exact Approximation , 2006, Scalable Optimization via Probabilistic Modeling.

[21]  Ivo Everts,et al.  Extended Compact Genetic Algorithm , 2004 .

[22]  Alexander Mendiburu,et al.  Parallel EDAs to create multivariate calibration models for quantitative chemical applications , 2006, J. Parallel Distributed Comput..

[23]  Kalyanmoy Deb,et al.  Analyzing Deception in Trap Functions , 1992, FOGA.

[24]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[25]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[26]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[27]  D. Goldberg,et al.  Don't evaluate, inherit , 2001 .

[28]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[29]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[30]  Kalyanmoy Deb,et al.  Sufficient conditions for deceptive and easy binary functions , 1994, Annals of Mathematics and Artificial Intelligence.

[31]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[32]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[33]  M. Troyer,et al.  Performance limitations of flat-histogram methods. , 2003, Physical review letters.

[34]  Josef Schwarz,et al.  The Parallel Bayesian Optimization Algorithm , 2000 .

[35]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[36]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[37]  David E. Goldberg,et al.  Hierarchical BOA Solves Ising Spin Glasses and MAXSAT , 2003, GECCO.

[38]  Martin Pelikan,et al.  Fitness Inheritance in the Bayesian Optimization Algorithm , 2004, GECCO.