Theory of Genetic Algorithms II: models for genetic operators over the string-tensor representation of populations and convergence to global optima for arbitrary fitness function under scaling

We present a theoretical framework for an asymptotically converging, scaled genetic algorithm which uses an arbitrary-size alphabet and common scaled genetic operators. The alphabet can be interpreted as a set of equidistant real numbers and multiple-spot mutation performs a scalable compromise between pure random search and neighborhood-based change on the alphabet level. We discuss several versions of the crossover operator and their interplay with mutation. In particular, we consider uniform crossover and gene-lottery crossover which does not commute with mutation. The Vose-Liepins version of mutation-crossover is also integrated in our approach. In order to achieve convergence to global optima, the mutation rate and the crossover rate have to be annealed to zero in proper fashion, and unbounded, power-law scaled proportional fitness selection is used with logarithmic growth in the exponent. Our analysis shows that using certain types of crossover operators and large population size allows for particularly slow annealing schedules for the crossover rate. In our discussion, we focus on the following three major aspects based upon contraction properties of the mutation and fitness selection operators: (i) the drive towards uniform populations in a genetic algorithm using standard operations, (ii) weak ergodicity of the inhomogeneous Markov chain describing the probabilistic model for the scaled algorithm, (iii) convergence to globally optimal solutions. In particular, we remove two restrictions imposed in Theorem 8.6 and Remark 8.7 of (Theoret. Comput. Sci. 259 (2001) 1) where a similar type of algorithm is considered as described here: mutation need not commute with crossover and the fitness function (which may come from a coevolutionary single species setting) need not have a single maximum.

[1]  Samir W. Mahfoud Finite Markov Chain Models of an Alternative Selection Strategy for the Genetic Algorithm , 1993, Complex Syst..

[2]  L. Shen,et al.  Linear Algebra , 1968 .

[3]  O. Catoni Sharp large deviations estimates for simulated annealing algorithms , 1991 .

[4]  Günter Rudolph,et al.  Finite Markov Chain Results in Evolutionary Computation: A Tour d'Horizon , 1998, Fundam. Informaticae.

[5]  Lothar M. Schmitt Optimization with genetic algorithms in multispecies environments , 2003, Proceedings Fifth International Conference on Computational Intelligence and Multimedia Applications. ICCIMA 2003.

[6]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[7]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[8]  Pedro Larrañaga,et al.  Genetic Algorithms: Bridging the Convergence Gap , 1999, Theor. Comput. Sci..

[9]  L. Darrell Whitley,et al.  An Executable Model of a Simple Genetic Algorithm , 1992, FOGA.

[10]  José Carlos Príncipe,et al.  A Simulated Annealing Like Convergence Theory for the Simple Genetic Algorithm , 1991, ICGA.

[11]  D. Fogel ASYMPTOTIC CONVERGENCE PROPERTIES OF GENETIC ALGORITHMS AND EVOLUTIONARY PROGRAMMING: ANALYSIS AND EXPERIMENTS , 1994 .

[12]  David E. Goldberg,et al.  An Analysis of Reproduction and Crossover in a Binary-Coded Genetic Algorithm , 1987, ICGA.

[13]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[14]  Awi Federgruen,et al.  Ergodicity in Parametric Nonstationary Markov Chains: An Application to Simulated Annealing Methods , 1987, Oper. Res..

[15]  Gábor Renner,et al.  Spline interpolation with genetic algorithms , 1997, Proceedings of 1997 International Conference on Shape Modeling and Applications.

[16]  Michael D. Vose,et al.  Modeling genetic algorithms with Markov chains , 1992, Annals of Mathematics and Artificial Intelligence.

[17]  Gary J. Koehler,et al.  A proof of the Vose-Liepins conjecture , 1994, Annals of Mathematics and Artificial Intelligence.

[18]  Siddhartha Bhattacharyya,et al.  An Analysis of Non-Binary Genetic Algorithms with Cardinality 2υ , 1994, Complex Syst..

[19]  Peter Nordin,et al.  The Effect of Extensive Use of the Mutation Operator on Generalization in Genetic Programming Using Sparse Data Sets , 1996, PPSN.

[20]  Joe Suzuki,et al.  A Further Result on the Markov Chain Model of Genetic Algorithms and Its Application to a Simulated Annealing-like Strategy , 1998, FOGA.

[21]  J. K. Lenstra,et al.  Local Search in Combinatorial Optimisation. , 1997 .

[22]  S. Tavaré,et al.  Computational Methods for the Coalescent , 1997 .

[23]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[24]  H. Geiringer On the Probability Theory of Linkage in Mendelian Heredity , 1944 .

[25]  Ehl Emile Aarts,et al.  Simulated annealing : an introduction , 1989 .

[26]  Riccardo Poli,et al.  Schema Theory for Genetic Programming with One-Point Crossover and Point Mutation , 1997, Evolutionary Computation.

[27]  Riccardo Poli,et al.  Exact Schema Theory for Genetic Programming and Variable-Length Genetic Algorithms with One-Point Crossover , 2001, Genetic Programming and Evolvable Machines.

[28]  Alden H. Wright,et al.  The Simple Genetic Algorithm and the Walsh Transform: Part II, The Inverse , 1998, Evolutionary Computation.

[29]  Michael D. Vose,et al.  Random Heuristic Search , 1999, Theor. Comput. Sci..

[30]  David E. Goldberg,et al.  A Note on Boltzmann Tournament Selection for Genetic Algorithms and Population-Oriented Simulated Annealing , 1990, Complex Syst..

[31]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[32]  Günter Rudolph,et al.  An Evolutionary Algorithm for Integer Programming , 1994, PPSN.

[33]  Michael D. Vose,et al.  The simple genetic algorithm - foundations and theory , 1999, Complex adaptive systems.

[34]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[35]  O. Catoni Rough Large Deviation Estimates for Simulated Annealing: Application to Exponential Schedules , 1992 .

[36]  David E. Goldberg,et al.  A Genetic Algorithm for Parallel Simulated Annealing , 1992, PPSN.

[37]  R. Cerf Asymptotic convergence of genetic algorithms , 1998, Advances in Applied Probability.

[38]  Lothar M. Schmitt,et al.  Theory of Coevolutionary Genetic Algorithms , 2003, ISPA.

[39]  Alden H. Wright,et al.  The Simple Genetic Algorithm and the Walsh Transform: Part I, Theory , 1998, Evolutionary Computation.

[40]  Christopher R. Stephens,et al.  Effective Degrees of Freedom in Genetic Algorithms and the Block Hypothesis , 1997, ICGA.

[41]  Lothar M. Schmitt Coevolutionary Convergence to Global Optima , 2003, GECCO.

[42]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[43]  Garrison W. Greenwood,et al.  Convergence in Evolutionary Programs with Self-Adaptation , 2001, Evolutionary Computation.

[44]  Lishan Kang,et al.  On the Convergence Rates of Genetic Algorithms , 1999, Theor. Comput. Sci..

[45]  S. Lang Complex Analysis , 1977 .

[46]  Dean Isaacson,et al.  Markov Chains: Theory and Applications , 1976 .

[47]  O. Catoni Applications of sharp large deviations estimates to optimal cooling schedules , 1991 .

[48]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[49]  Siddhartha Bhattacharyya,et al.  General Cardinality Genetic Algorithms , 1997, Evolutionary Computation.

[50]  A. Wright,et al.  Form Invariance and Implicit Parallelism , 2001, Evolutionary Computation.

[51]  Alexandru Agapie,et al.  Theoretical Analysis of Mutation-Adaptive Evolutionary Algorithms , 2001, Evolutionary Computation.

[52]  E. Seneta Non-negative Matrices and Markov Chains (Springer Series in Statistics) , 1981 .

[53]  Tatsuya Nomura,et al.  An Analysis of Two-Parent Recombinations for Real-Valued Chromosomes in an Infinite Population , 2001, Evolutionary Computation.

[54]  David E. Goldberg,et al.  Parallel Recombinative Simulated Annealing: A Genetic Algorithm , 1995, Parallel Comput..

[55]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[56]  Melanie Mitchell,et al.  An introduction to genetic algorithms , 1996 .

[57]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[58]  Haldun Aytug,et al.  Stopping Criteria for Finite Length Genetic Algorithms , 1996, INFORMS J. Comput..

[59]  H. H. Schaefer Banach Lattices and Positive Operators , 1975 .

[60]  Raphaël Cerf,et al.  An Asymptotic Theory of Genetic Algorithms , 1995, Artificial Evolution.

[61]  Gunar E. Liepins,et al.  Punctuated Equilibria in Genetic Search , 1991, Complex Syst..

[62]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[63]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[64]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (2nd, extended ed.) , 1994 .

[65]  RudolphGünter Finite Markov chain results in evolutionary computation , 1998 .

[66]  Lothar M. Schmitt,et al.  Theory of genetic algorithms , 2001, Theor. Comput. Sci..

[67]  Chrystopher L. Nehaniv,et al.  Linear Analysis of Genetic Algorithms , 1998, Theor. Comput. Sci..

[68]  José Carlos Príncipe,et al.  A Markov Chain Framework for the Simple Genetic Algorithm , 1993, Evolutionary Computation.