A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning

Genetic algorithms (GAs) are biologically motivated adaptive systems which have been used, with varying degrees of success, for function optimization. In this study, an abstraction of the basic genetic algorithm, the Equilibrium Genetic Algorithm (EGA), and the GA in turn, are reconsidered within the framework of competitive learning. This new perspective reveals a number of different possibilities for performance improvements. This paper explores population-based incremental learning (PBIL), a method of combining the mechanisms of a generational genetic algorithm with simple competitive learning. The combination of these two methods reveals a tool which is far simpler than a GA, and which out-performs a GA on large set of optimization problems in terms of both speed and accuracy. This paper presents an empirical analysis of where the proposed technique will outperform genetic algorithms, and describes a class of problems in which a genetic algorithm may be able to perform better. Extensions to this algorithm are discussed and analyzed. PBIL and extensions are compared with a standard GA on twelve problems, including standard numerical optimization functions, traditional GA test suite problems, and NP-Complete problems.

[1]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[3]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[4]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[5]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[6]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[7]  David H. Ackley,et al.  A Connectionist Algorithm for Genetic Search , 1985, ICGA.

[8]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[9]  D. Ackley.  A connectionist machine for genetic hillclimbing , 1987 .

[10]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[11]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[12]  J. David Schaffer,et al.  Representation and Hidden Bias: Gray vs. Binary Coding for Genetic Algorithms , 1988, ML.

[13]  L. D. Whitley,et al.  Scheduling Problems and Traveling Salesmen: The Genetic Edge Recombination Operator , 1989, ICGA.

[14]  Heinz Mühlenbein,et al.  Parallel Genetic Algorithms, Population Genetics, and Combinatorial Optimization , 1989, Parallelism, Learning, Evolution.

[15]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[16]  V. Rich.  Personal communication , 1989, Nature.

[17]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[18]  L. Darrell Whitley,et al.  Optimizing Neural Networks Using FasterMore Accurate Genetic Search , 1989, ICGA.

[19]  S. Fahlman.  Fast-learning variations on back propagation: an empirical study. , 1989 .

[20]  L. Darrell Whitley,et al.  GENITOR II: a distributed genetic algorithm , 1990, J. Exp. Theor. Artif. Intell..

[21]  Teuvo Kohonen,et al.  Improved versions of learning vector quantization , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[22]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[23]  GUNAR E. LIEPINS,et al.  Representational issues in genetic optimization , 1990, J. Exp. Theor. Artif. Intell..

[24]  Martina Gorges-Schleuter,et al.  Explicit Parallelism of Genetic Algorithms through Population Structures , 1990, PPSN.

[25]  Lawrence Davis,et al.  Bit-Climbing, Representational Bias, and Test Suite Design , 1991, ICGA.

[26]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[27]  Dana S. Richards,et al.  Distributed genetic algorithms for the floorplan design problem , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[28]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[29]  Gilbert Syswerda,et al.  Simulated Crossover in Genetic Algorithms , 1992, FOGA.

[30]  Kennetb A. De.  Genetic Algorithms Are NOT Function Optimizers , 1992 .

[31]  William M. Spears,et al.  Crossover or Mutation? , 1992, FOGA.

[32]  L. Darrell Whitley,et al.  International Workshop on Combinations of Genetic Algorithms and Neural Networks , 1992 .

[33]  John J. Grefenstette,et al.  Deception Considered Harmful , 1992, FOGA.

[34]  Shumeet Baluja,et al.  Structure and Performance of Fine-Grain Parallelism in Genetic Search , 1993, ICGA.

[35]  Takeshi Yamada,et al.  The ECOlogical Framework II : Improving GA Performance At Virtually Zero Cost , 1993, ICGA.

[36]  Leslie Pack Kaelbling,et al.  Learning in embedded systems , 1993 .

[37]  Thomas Bäck,et al.  Optimal Mutation Rates in Genetic Search , 1993, ICGA.

[38]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[39]  Peter Ross,et al.  A Promising Genetic Algorithm Approach to Job-Shop SchedulingRe-Schedulingand Open-Shop Scheduling Problems , 1993, ICGA.

[40]  Daniel Polani,et al.  Training Kohonen Feature Maps in Different Topologies: An Analysis Using Genetic Algorithms , 1993, ICGA.

[41]  Larry J. Eshelman,et al.  Crossover's Niche , 1993, ICGA.

[42]  L. Darrell Whitley,et al.  Serial and Parallel Genetic Algorithms as Function Optimizers , 1993, ICGA.

[43]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[44]  Frédéric Gruau,et al.  Genetic Synthesis of Modular Neural Networks , 1993, ICGA.

[45]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[46]  L. C. Stayton,et al.  On the effectiveness of crossover in simulated evolutionary optimization. , 1994, Bio Systems.