Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning

Abstract We present a theory of population based optimization methods using approximations of search distributions. We prove convergence of the search distribution to the global optima for the factorized distribution algorithm (FDA) if the search distribution is a Boltzmann distribution and the size of the population is large enough. Convergence is defined in a strong sense––the global optima are attractors of a dynamical system describing mathematically the algorithm. We investigate an adaptive annealing schedule and show its similarity to truncation selection. The inverse temperature β is changed inversely proportionally to the standard deviation of the population. We extend FDA by using a Bayesian hyper-parameter. The hyper-parameter is related to mutation in evolutionary algorithms. We derive an upper bound on the hyper-parameter to ensure that FDA still generates the optima with high probability. We discuss the relation of the FDA approach to methods used in statistical physics to approximate a Boltzmann distribution and to belief propagation in probabilistic reasoning. In the last part are sparsely connected. Our empirical results are as good or even better than any other method used for this problem.

[1]  Michael I. Jordan Learning in Graphical Models , 1999, NATO ASI Series.

[2]  Pedro Larrañaga,et al.  Benefits of Data Clustering in Multimodal Function Optimization via EDAs , 2002, Estimation of Distribution Algorithms.

[3]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[4]  S.,et al.  An Efficient Heuristic Procedure for Partitioning Graphs , 2022 .

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  T. Mahnig,et al.  A new adaptive Boltzmann selection schedule SDS , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[7]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[8]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[9]  Bernd Freisleben,et al.  Fitness Landscapes, Memetic Algorithms, and Greedy Operators for Graph Bipartitioning , 2000, Evolutionary Computation.

[10]  A. Prügel-Bennett,et al.  The dynamics of a genetic algorithm for simple random Ising systems , 1997 .

[11]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[12]  El-Ghazali Talbi,et al.  A parallel genetic algorithm for the graph partitioning problem , 1991, ICS '91.

[13]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[16]  Charles M. Fiduccia,et al.  A linear-time heuristic for improving network partitions , 1988, 25 years of DAC.

[17]  T. Mahnig,et al.  Evolutionary algorithms: from recombination to search distributions , 2001 .

[18]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[19]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[20]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[21]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[22]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[23]  Heinz Mühlenbein,et al.  Evolutionary computation and Wright's equation , 2002, Theor. Comput. Sci..

[24]  D. Sherrington Stochastic Processes in Physics and Chemistry , 1983 .

[25]  Yoshinori Uesaka,et al.  Foundations of real-world intelligence , 2001 .

[26]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[27]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[28]  P. Stadler Fitness Landscapes , 1993 .

[29]  J. R. Newman The World of Mathematics , 1961 .

[30]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[31]  Pedro Larrañaga,et al.  Combinatonal Optimization by Learning and Simulation of Bayesian Networks , 2000, UAI.

[32]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[33]  H. Geiringer On the Probability Theory of Linkage in Mendelian Heredity , 1944 .

[34]  Yong-Hyuk Kim,et al.  A Hybrid Genetic Search for Graph Partitioning Based on Lock Gain , 2000, GECCO.