Parallel implementation of EDAs based on probabilistic graphical models

This paper proposes new parallel versions of some estimation of distribution algorithms (EDAs). Focus is on maintenance of the behavior of sequential EDAs that use probabilistic graphical models (Bayesian networks and Gaussian networks), implementing a master-slave workload distribution for the most computationally intensive phases: learning the probability distribution and, in one algorithm, "sampling and evaluation of individuals." In discrete domains, we explain the parallelization of EBNA/sub BIC/ and EBNA/sub PC/ algorithms, while in continuous domains, the selected algorithms are EGNA/sub BIC/ and EGNA/sub EE/. Implementation has been done using two APIs: message passing interface and POSIX threads. The parallel programs can run efficiently on a range of target parallel computers. Experiments to evaluate the programs in terms of speed up and efficiency have been carried out on a cluster of multiprocessors. Compared with the sequential versions, they show reasonable gains in terms of speed.

[1]  Josef Schwarz,et al.  The Parallel Bayesian Optimization Algorithm , 2000 .

[2]  Michèle Sebag,et al.  Extending Population-Based Incremental Learning to Continuous Search Spaces , 1998, PPSN.

[3]  Jiri Ocenasek,et al.  THE DISTRIBUTED BAYESIAN OPTIMIZATION ALGORITHM FOR COMBINATORIAL OPTIMIZATION , 2001 .

[4]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[5]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[6]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[7]  David E. Goldberg,et al.  Multiple-Deme Parallel Estimation of Distribution Algorithms: Basic Framework and Application , 2003, PPAM.

[8]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[9]  Pedro Larrañaga,et al.  Mathematical Modeling of Discrete Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[10]  Qingfu Zhang,et al.  On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm , 2004, IEEE Transactions on Evolutionary Computation.

[11]  Pedro Larrañaga,et al.  Parallel Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[12]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[13]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[14]  P. Spirtes,et al.  An Algorithm for Fast Recovery of Sparse Causal Graphs , 1991 .

[15]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[16]  A. A. Zhigli︠a︡vskiĭ,et al.  Theory of Global Random Search , 1991 .

[17]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[18]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[19]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[20]  David E. Goldberg,et al.  Hierarchical Problem Solving and the Bayesian Optimization Algorithm , 2000, GECCO.

[21]  Martin Pelikan,et al.  Design of Multithreaded Estimation of Distribution Algorithms , 2003, GECCO.

[22]  Joe Whittaker,et al.  Edge Exclusion Tests for Graphical Gaussian Models , 1999, Learning in Graphical Models.

[23]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[24]  D. Goldberg,et al.  Research on the Bayesian Optimization Algorithm Research on the Bayesian Optimization Algorithm , 2022 .

[25]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[26]  BayesiannetworksPedro,et al.  Combinatorial optimization by learning and simulation of , 2000 .

[27]  Pedro Larrañaga,et al.  Optimization in Continuous Domains by Learning and Simulation of Gaussian Networks , 2000 .

[28]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[29]  David E. Goldberg,et al.  Genetic Algorithms, Clustering, and the Breaking of Symmetry , 2000, PPSN.

[30]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[31]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[32]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[33]  Qingfu Zhang,et al.  On the convergence of a class of estimation of distribution algorithms , 2004, IEEE Transactions on Evolutionary Computation.