Effective Linkage Learning Using Low-Order Statistics and Clustering

The adoption of probabilistic models for selected individuals is a powerful approach for evolutionary computation. Probabilistic models based on high-order statistics have been used by estimation of distribution algorithms (EDAs), resulting better effectiveness when searching for global optima for hard optimization problems. This paper proposes a new framework for evolutionary algorithms, which combines a simple EDA based on order 1 statistics and a clustering technique in order to avoid the high computational cost required by higher order EDAs. The algorithm uses clustering to group genotypically similar solutions, relying that different clusters focus on different substructures and the combination of information from different clusters effectively combines substructures. The combination mechanism uses an information gain measure when deciding which cluster is more informative for any given gene position, during a pairwise cluster combination. Empirical evaluations effectively cover a comprehensive range of benchmark optimization problems.

[1]  D. Goldberg,et al.  A Survey of Linkage Learning Techniques in Genetic and Evolutionary Algorithms , 2007 .

[2]  David E. Goldberg,et al.  Linkage learning, overlapping building blocks, and systematic strategy for scalable recombination , 2005, GECCO '05.

[3]  Ben Hui Liu,et al.  Statistical Genomics: Linkage, Mapping, and QTL Analysis , 1997 .

[4]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[5]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[6]  D. Goldberg,et al.  Sporadic Model Building for Efficiency Enhancement of hBOA Martin Pelikan , 2005 .

[7]  Dirk Thierens,et al.  Advancing continuous IDEAs with mixture distributions and factorization selection metrics , 2001 .

[8]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[9]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[10]  David E. Goldberg,et al.  Genetic Algorithms, Clustering, and the Breaking of Symmetry , 2000, PPSN.

[11]  G. Harik.  Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms , 1997 .

[12]  Marcus Gallagher,et al.  On the importance of diversity maintenance in estimation of distribution algorithms , 2005, GECCO '05.

[13]  David E. Goldberg,et al.  Sporadic model building for efficiency enhancement of hierarchical BOA , 2006, GECCO.

[14]  Pedro Larrañaga,et al.  Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks , 2005, Evolutionary Computation.

[15]  Jordan B. Pollack,et al.  Modeling Building-Block Interdependency , 1998, PPSN.

[16]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[17]  Qingfu Zhang,et al.  An evolutionary algorithm with guided mutation for the maximum clique problem , 2005, IEEE Transactions on Evolutionary Computation.

[18]  Aurora Trinidad Ramirez Pozo,et al.  An Incremental Approach for Niching and Building Block Detection via Clustering , 2007, Seventh International Conference on Intelligent Systems Design and Applications (ISDA 2007).

[19]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[20]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[21]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[22]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[23]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[24]  David E. Goldberg,et al.  Multiple-Deme Parallel Estimation of Distribution Algorithms: Basic Framework and Application , 2003, PPAM.

[25]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[26]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[27]  Fernando G. Lobo,et al.  A parameter-less genetic algorithm , 1999, GECCO.

[28]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[29]  Pedro Larrañaga,et al.  Exact Bayesian network learning in estimation of distribution algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[30]  A. Agresti,et al.  Approximate is Better than “Exact” for Interval Estimation of Binomial Proportions , 1998 .

[31]  J. A. Lozano,et al.  Analyzing the PBIL Algorithm by Means of Discrete Dynamical Systems , 2000 .

[32]  Chang Wook Ahn,et al.  Clustering-Based Probabilistic Model Fitting in Estimation of Distribution Algorithms , 2006, IEICE Trans. Inf. Syst..

[33]  Ivan Bratko,et al.  Testing the significance of attribute interactions , 2004, ICML.

[34]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[35]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[36]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[37]  David E. Goldberg,et al.  The Race, the Hurdle, and the Sweet Spot , 1998 .

[38]  Judea Pearl,et al.  MARKOV AND BAYESIAN NETWORKS: Two Graphical Representations of Probabilistic Knowledge , 1988 .

[39]  Qingfu Zhang,et al.  On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm , 2004, IEEE Transactions on Evolutionary Computation.