Combinations of genetic algorithms and neural networks: a survey of the state of the art

Various schemes for combining genetic algorithms and neural networks have been proposed and tested in recent years, but the literature is scattered among a variety of journals, proceedings and technical reports. Activity in this area is clearly increasing. The authors provide an overview of this body of literature drawing out common themes and providing, where possible, the emerging wisdom about what seems to work and what does not.<<ETX>>

[1]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[2]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[3]  John Daniel. Bagley,et al.  The behavior of adaptive systems which employ genetic and correlation algorithms : technical report , 1967 .

[4]  J. Reed,et al.  Simulation of biological evolution and machine learning. I. Selection of self-reproducing numeric patterns by data processing machines, effects of hereditary control, mutation type and crossing. , 1967, Journal of theoretical biology.

[5]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[8]  E. Hansen.  Numerical Optimization of Computer Models (Hans-Paul Schwefel) , 1983 .

[9]  Ingo Rechenberg,et al.  The Evolution Strategy. A Mathematical Model of Darwinian Evolution , 1984 .

[10]  A. J. Fenanzo,et al.  Darwinian evolution as a paradigm for AI research , 1986, SGAR.

[11]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[12]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[13]  J. R. Knisley,et al.  A Darwinian approach to artificial neural systems , 1987 .

[14]  M. Dyer,et al.  Toward the Evolution of Symbols , 1987, ICGA.

[15]  David H. Ackley,et al.  An empirical study of bit vector function optimization , 1987 .

[16]  Charles W. Anderson,et al.  Strategy Learning with Multilayer Connectionist Representations , 1987 .

[17]  R. Lippmann,et al.  An introduction to computing with neural nets , 1987, IEEE ASSP Magazine.

[18]  Aviv Bergman,et al.  BREEDING INTELLIGENT AUTOMATA. , 1987 .

[19]  Eric Mjolsness,et al.  A preliminary analysis of recursively generated networks , 1987 .

[20]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[21]  D. Ackley.  A connectionist machine for genetic hillclimbing , 1987 .

[22]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[23]  Darrell Whitley,et al.  Genitor: a different genetic algorithm , 1988 .

[24]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[25]  T. P. Caudell.  Parametric connectivity: feasibility of learning in constrained weight space , 1989, International 1989 Joint Conference on Neural Networks.

[26]  Lawrence Davis,et al.  Mapping Neural Networks into Classifier Systems , 1989, ICGA.

[27]  Thomas P. Caudell,et al.  Parametric Connectivity: Training of Constrained Networks using Genetic Algorithms , 1989, ICGA.

[28]  Richard K. Belew,et al.  When Both Individuals and Populations Search: Adding Simple Learning to the Genetic Algorithm , 1989, ICGA.

[29]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[30]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[31]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[32]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[33]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[34]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[35]  Tariq Samad,et al.  Designing Application-Specific Neural Networks Using the Genetic Algorithm , 1989, NIPS.

[36]  Darrell Whitley,et al.  Applying genetic algorithms to neural network learning , 1989 .

[37]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[38]  David H. Sharp,et al.  Scaling, machine learning, and genetic neural nets , 1989 .

[39]  Hugo de Garis.  Brain Building with GenNets , 1990 .

[40]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[41]  Michael Scholz.  A Learning Strategy for Neural Networks Based on a Modified Evolutionary Strategy , 1990, PPSN.

[42]  George N. Saridis,et al.  A Boltzmann machine for the organization of intelligent machines , 1990, IEEE Trans. Syst. Man Cybern..

[43]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[44]  Lyle H. Ungar,et al.  A bioreactor benchmark for adaptive network-based process control , 1990 .

[45]  Peter M. Todd,et al.  Exploring Adaptive Agency III: Simulating the Evolution of Habituation and Sensitization , 1990, PPSN.

[46]  Richard Lippmann,et al.  Using Genetic Algorithms to Improve Pattern Classification Performance , 1990, NIPS.

[47]  Wolfram Schiffmann,et al.  Performance Evaluation of Evolutionarily Created Neural Network Topologies , 1990, PPSN.

[48]  Larry J. Eshelman,et al.  Spurious Correlations and Premature Convergence in Genetic Algorithms , 1990, FOGA.

[49]  Kendall E. Nygard,et al.  Modular neural networks and distributed adaptive search for traveling salesman algorithms , 1990, Defense, Security, and Sensing.

[50]  D. B. Fogel,et al.  Neural network techniques for navigation of AUVs , 1990, Symposium on Autonomous Underwater Vehicle Technology.

[51]  Hiroaki Kitano,et al.  Empirical Studies on the Speed of Convergence of Neural Network Training Using Genetic Algorithms , 1990, AAAI.

[52]  Stewart W. Wilson.  Perceptron redux: emergence of structure , 1990 .

[53]  Hugo de Garis.  Genetic Programming: Evolution of a Time Dependent Neural Network Module which Teaches a Pair of Stick Legs to Walk , 1990, ECAI.

[54]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[55]  Randall Beer,et al.  Intelligence as Adaptive Behavior , 1990 .

[56]  David G. Stork,et al.  Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution , 1990, NIPS 1990.

[57]  Richard K. Belew,et al.  Evolution, Learning, and Culture: Computational Metaphors for Adaptive Algorithms , 1990, Complex Syst..

[58]  Larry J. Eshelman,et al.  Using genetic search to exploit the emergent behavior of neural networks , 1990 .

[59]  W. Thomas Miller,et al.  A challenging set of control problems , 1990 .

[60]  W. Dress.  Electronic life and synthetic intelligent systems , 1990 .

[61]  Alfred Ultsch,et al.  Genetic Improvements of Feedforward Nets for Approximating Functions , 1990, PPSN.

[62]  H. de Garis.  Genetic programming: building nanobrains with genetically programmed neural network modules , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[63]  Leslie S. Smith,et al.  GANNET: Genetic Design of a Neural Net for Face Recognition , 1990, PPSN.

[64]  Gunar E. Liepins,et al.  Deceptiveness and Genetic Algorithm Dynamics , 1990, FOGA.

[65]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[66]  W. Pinebrook.  The evolution of strategy. , 1990, Case studies in health administration.

[67]  David R. Jefferson,et al.  An Artificial Neural Network Representation for Artificial Organisms , 1990, PPSN.

[68]  Kendall E. Nygard,et al.  Improving the Performance of Genetic Algorithms in Automated Discovery of Parameters , 1990, ML.

[69]  Jochen Heistermann.  The Application of a Genetic Approach as an Algorithm for Neural Networks , 1990, PPSN.

[70]  Stefan C. Kremer,et al.  The Guelph Darwin Project: the evolution of neural networks by genetic algorithms , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[71]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning with Multilayer Neural Networks , 1991, ICGA.

[72]  Alexis P. Wieland,et al.  Evolving Controls for Unstable Systems , 1991 .

[73]  Paul Juell,et al.  Integration of adaptive machine learning and knowledge-based systems for routing and scheduling applications , 1991 .

[74]  Nigel Steele,et al.  Genetic algorithms and the design of artificial neural networks , 1991 .

[75]  Maureen Caudill,et al.  Evolutionary neural networks , 1991 .

[76]  Emergent computation , 1991 .

[77]  Ryszard S. Michalski,et al.  Comparing learning paradigms via diagrammatic visualization: a case study in single concept learning using symbolic, neural net and genetic algorithm methods , 1991 .

[78]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[79]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[80]  Larry J. Eshelman,et al.  On Crossover as an Evolutionarily Viable Strategy , 1991, ICGA.

[81]  Jean-Arcady Meyer,et al.  Exploring Adaptive Agency II: Simulating the Evolution of Associative Learning , 1991 .

[82]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[83]  Ron Meir,et al.  Evolving a learning algorithm for the binary perceptron , 1991 .

[84]  L. Darrell Whitley,et al.  The Only Challenging Problems Are Deceptive: Global Search by Solving Order-1 Hyperplanes , 1991, ICGA.

[85]  Lawrence Davis,et al.  Hybridizing the Genetic Algorithm and the K Nearest Neighbors Classification Algorithm , 1991, ICGA.

[86]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[87]  Peter M. Todd,et al.  Exploring adaptive agency II: simulating the evolution of associative learning , 1991 .

[88]  D. B. Fogel,et al.  Using evolutionary neural networks for arterial waveform discrimination , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[89]  T. Samad,et al.  Genetic optimization of self-organizing feature maps , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[90]  Melanie Mitchell,et al.  The Performance of Genetic Algorithms on Walsh Polynomials: Some Anomalous Results and Their Explanation , 1991, ICGA.

[91]  Russell C. Eberhart,et al.  Designing neural network explanation facilities using genetic algorithms , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[92]  David H. Ackley,et al.  Adaptation in Constant Utility Non-Stationary Environments , 1991, ICGA.

[93]  Hillol Kargupta,et al.  System Identification with Evolving Polynomial Networks , 1991, ICGA.

[94]  Jan Torreele,et al.  Temporal Processing with Recurrent Networks: An Evolutionary Approach , 1991, ICGA.

[95]  Yukinori Kakazu,et al.  An Approach to the Analysis of the Basins of the Associative Memory Model Using Genetic Algorithms , 1991, ICGA.

[96]  L. Marti.  Genetically generated neural networks. II. Searching for an optimal representation , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[97]  Constantinos S. Pattichis,et al.  Neural networks, genetic algorithms and the K-means algorithm: in search of data classification , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[98]  J. Gonzalez-Seco,et al.  A genetic algorithm as the learning procedure for neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[99]  T. P. Caudell,et al.  Genetic algorithms as a tool for the analysis of adaptive resonance theory network training sets , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[100]  Donald E. Brown,et al.  Fast generic selection of features for neural network classifiers , 1992, IEEE Trans. Neural Networks.

[101]  Darrell Whitley,et al.  Genetic cascade learning for neural networks , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[102]  L. Marti,et al.  Genetically generated neural networks-I: representational effects , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[103]  Nicholas J. Radcliffe,et al.  Genetic neural networks on MIMD computers , 1992 .

[104]  Loke Soo Hsu,et al.  Input pattern encoding through generalized adaptive search , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[105]  J. G. Elias.  Genetic generation of connection patterns for a dynamic artificial neural network , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[106]  Stefan Bornholdt,et al.  General asymmetric neural networks and structure design by genetic algorithms: a learning rule for temporal patterns , 1992, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[107]  K. KrishnaKumar,et al.  Immunized neurocontrol-concepts and initial results , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[108]  Darrell Whitley,et al.  Genetic sparse distributed memory , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[109]  Mitchell A. Potter,et al.  A genetic cascade-correlation learning algorithm , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[110]  R. C. Eberhart,et al.  The role of genetic algorithms in neural network query-based learning and explanation facilities , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[111]  W. Rudnick.  Genetic algorithms and fitness variance with an application to the automated design of artificial neural networks , 1992 .

[112]  R. Shonkwiler,et al.  Genetic algorithm/neural network synergy for nonlinearly constrained optimization problems , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[113]  Yoshiaki Ichikawa,et al.  Neural network application for direct feedback controllers , 1992, IEEE Trans. Neural Networks.

[114]  Randall D. Beer,et al.  Evolving Dynamical Neural Networks for Adaptive Behavior , 1992, Adapt. Behav..

[115]  Frédéric Gruau,et al.  Genetic synthesis of Boolean neural networks with a cell rewriting developmental process , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[116]  K. Lindgren,et al.  Regular language inference using evolving neural networks , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[117]  R. E. Uhrig,et al.  Using genetic algorithms to select inputs for neural networks , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[118]  James F. Frenzel,et al.  Training product unit neural networks with genetic algorithms , 1993, IEEE Expert.

[119]  Jeffrey L. Elman,et al.  Learning and Evolution in Neural Networks , 1994, Adapt. Behav..