Spurious dependencies and EDA scalability

Numerous studies have shown that advanced estimation of distribution algorithms (EDAs) often discover spurious (unnecessary) dependencies. Nonetheless, only little prior work exists that would study the effects of spurious dependencies on EDA performance. This paper examines the effects of spurious dependencies on the performance and scalability of EDAs with the main focus on EDAs with marginal product models and the onemax problem. A theoretical model is proposed to analyze the effects of spurious dependencies on the population sizing in EDAs and the theory is verified with experiments. The effects of spurious dependencies on the number of generations are studied empirically. The effects of replacement strategies on the performance of EDAs with spurious linkage are also investigated.

[1]  Martin Pelikan,et al.  From mating pool distributions to model overfitting , 2008, GECCO '08.

[2]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[3]  Thomas D. LaToza,et al.  On the supply of building blocks , 2001 .

[4]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[5]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[6]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[7]  David E. Goldberg,et al.  Scalability of the Bayesian optimization algorithm , 2002, Int. J. Approx. Reason..

[8]  John H. Holland,et al.  Genetic Algorithms and the Optimal Allocation of Trials , 1973, SIAM J. Comput..

[9]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[10]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[11]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[12]  Shumeet Baluja,et al.  Using Optimal Dependency-Trees for Combinational Optimization , 1997, ICML.

[13]  E. Cantu-Paz,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1997, Evolutionary Computation.

[14]  Ivo Everts,et al.  Extended Compact Genetic Algorithm , 2004 .

[15]  Martin V. Butz,et al.  Substructural Neighborhoods for Local Search in the Bayesian Optimization Algorithm , 2006, PPSN.

[16]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[17]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[18]  Pedro Larrañaga,et al.  Interactions and dependencies in estimation of distribution algorithms , 2005, 2005 IEEE Congress on Evolutionary Computation.

[19]  Dirk Thierens,et al.  Scalability Problems of Simple Genetic Algorithms , 1999, Evolutionary Computation.

[20]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[21]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[22]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[23]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[24]  David E. Goldberg,et al.  Toward an Understanding of the Quality and Efficiency of Model Building for Genetic Algorithms , 2004, GECCO.

[25]  Franz Rothlauf,et al.  Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms , 2004 .

[26]  Fernando G. Lobo,et al.  Extended Compact Genetic Algorithm in C , 1999 .

[27]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[28]  David E. Goldberg,et al.  Population sizing for entropy-based model building in discrete estimation of distribution algorithms , 2007, GECCO '07.

[29]  Hao Wu,et al.  Does overfitting affect performance in estimation of distribution algorithms , 2006, GECCO.

[30]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[31]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[32]  David E. Goldberg,et al.  Linkage learning, overlapping building blocks, and systematic strategy for scalable recombination , 2005, GECCO '05.

[33]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[34]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[35]  David E. Goldberg,et al.  Influence of selection and replacement strategies on linkage learning in BOA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[36]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[37]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .