Genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems

Many seemingly different problems in artificial intelligence, symbolic processing, and machine learning can be viewed as requiring discovery of a computer program that produces some desired output for particular inputs. When viewed in this way, the process of solving these problems becomes equivalent to searching a space of possible computer programs for a most fit individual computer program. The new "genetic programming" paradigm described herein provides a way to search for this most fit individual computer program. In this new "genetic programming" paradigm, populations of computer programs are genetically bred using the Darwinian principle of survival of the fittest and using a genetic crossover (recombination) operator appropriate for genetically mating computer programs. In this paper, the process of formulating and solving problems using this new paradigm is illustrated using examples from various areas. Examples come from the areas of machine learning of a function; planning; sequence induction; function function identification (including symbolic regression, empirical discovery, "data to function" symbolic integration, "data to function" symbolic differentiation); solving equations, including differential equations, integral equations, and functional equations); concept formation; automatic programming; pattern recognition, time-optimal control; playing differential pursuer-evader games; neural network design; and finding a game-playing strategyfor a discrete game in extensive form.

[1]  Richard M. Friedberg,et al.  A Learning Machine: Part I , 1958, IBM J. Res. Dev..

[2]  Richard M. Friedberg,et al.  A Learning Machine: Part II , 1959, IBM J. Res. Dev..

[3]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[4]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[5]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[6]  N. S. Barnett,et al.  Private communication , 1969 .

[7]  David R. Barstow,et al.  Progress report on program-understanding systems. , 1974 .

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Douglas B. Lenat,et al.  AM, an artificial intelligence approach to discovery in mathematics as heuristic search , 1976 .

[10]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[11]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[12]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[13]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Stephen F. Smith,et al.  Flexible Learning of Problem Solving Heuristics Through Adaptive Search , 1983, IJCAI.

[15]  Douglas B. Lenat,et al.  Why AM and EURISKO Appear to Work , 1984, Artif. Intell..

[16]  Douglas B. Lenat,et al.  The Role of Heuristics in Learning by Discovery: Three Case Studies , 1983 .

[17]  John R. Anderson,et al.  Machine learning - an artificial intelligence approach , 1982, Symbolic computation.

[18]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[19]  John J. Grefenstette,et al.  Genetic algorithms and their applications , 1987 .

[20]  Thomas H. Westerdale,et al.  The Bucket Brigade Is Not Genetic , 1985, International Conference on Genetic Algorithms.

[21]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[22]  P. Anandan,et al.  Cooperativity in Networks of Pattern Recognizing Stochastic Learning Automata , 1986 .

[23]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[24]  Paul S. Rosenbloom,et al.  Universal Subgoaling and Chunking , 1986 .

[25]  Kumpati S. Narendra,et al.  Adaptive and Learning Systems , 1986 .

[26]  湯浅 太一,et al.  Common Lisp(パネル討論会) , 1986 .

[27]  John Dickinson,et al.  Using the Genetic Algorithm to Generate LISP Source Code to Solve the Prisoner's Dilemma , 1987, ICGA.

[28]  Michael R. Genesereth,et al.  Logical foundations of artificial intelligence , 1987 .

[29]  Herbert A. Simon,et al.  Scientific discovery: compulalional explorations of the creative process , 1987 .

[30]  Clive Richards,et al.  The Blind Watchmaker , 1987, Bristol Medico-Chirurgical Journal.

[31]  R. D. Mills,et al.  Using a Small Algebraic Manipulation System to Solve Differential and Integral Equations by Variational and Approximation Techniques , 1987, J. Symb. Comput..

[32]  Kenneth A. De Jong,et al.  On Using Genetic Algorithms to Search Program Spaces , 1987, ICGA.

[33]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[34]  Stewart W. Wilson.  Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[35]  Herbert A. Simon,et al.  Scientific Discovery: Computational Explorations of the Creative Processes , 1987 .

[36]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[37]  J. Ross Quinlan,et al.  An Empirical Comparison of Genetic and Decision-Tree Classifiers , 1988, ML.

[38]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[39]  Stewart W. Wilson.  Bid Competition and Specificity Reconsidered , 1988, Complex Syst..

[40]  A. Magurran.  Ecological Diversity and Its Measurement , 1988, Springer Netherlands.

[41]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[42]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[43]  Richard D. Porter,et al.  M2 per unit of potential GNP as an anchor for the price level , 1989 .

[44]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[45]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[46]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[47]  Paul Thagard,et al.  Induction: Processes Of Inference , 1989 .

[48]  S.C.J. Garth,et al.  Neural networks for artificial intelligence , 1989 .

[49]  T. M. Humphrey.  Precursors of the P-Star Model , 1989 .

[50]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[51]  Jan M. Zytkow,et al.  Data-Driven Approaches to Empirical Discovery , 1989, Artif. Intell..

[52]  John R. Koza,et al.  Hierarchical Genetic Algorithms Operating on Populations of Computer Programs , 1989, IJCAI.

[53]  John R. Koza,et al.  Concept Formation and Decision Tree Induction Using the Genetic Programming Paradigm , 1990, PPSN.

[54]  John R. Koza,et al.  Genetic breeding of non-linear optimal control strategies for broom balancing , 1990 .

[55]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[56]  Stuart L. Anderson,et al.  Random Number Generators on Vector Supercomputers and Other Advanced Architectures , 1990, SIAM Rev..

[57]  Stephanie Forrest,et al.  Parallelism and programming in classifier systems , 1990 .

[58]  M. Sarrafzadeh.  Department of electrical engineering and computer science , 1990, SIGD.

[59]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[60]  Ryszard S. Michalski,et al.  Machine learning: an artificial intelligence approach volume III , 1990 .

[61]  Matthew Zeidenberg,et al.  Neural networks in artificial intelligence , 1990, Ellis Horwood series in artificial intelligence.

[62]  John R. Koza,et al.  Evolution and co-evolution of computer programs to control independently-acting agents , 1991 .

[63]  John R. Koza,et al.  Genetic evolution and co-evolution of computer programs , 1991 .

[64]  Mitchel Resnick,et al.  Animal simulations with Logo: Massive parallelism for the masses , 1991 .

[65]  Luc Steels,et al.  Towards a theory of emergent functionality , 1991 .

[66]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[67]  John R. Koza,et al.  A GENETIC APPROACH TO ECONOMETRIC MODELING , 1992 .