Evolutionary neural trees for modeling and predicting complex systems

Abstract Modeling and predicting the behavior of many technical systems is complicated because they are generally characterized by a large number of variables, parameters and interactions, and limited amounts of collected data. This paper investigates an evolutionary method for learning models of such systems. The models thus evolved are based on trees of heterogeneous neural units. The set of different neuron types is defined by the application domain, and the specific type of each unit is determined during the evolutionary learning process. The structure, size, and weights of the neural trees are also adapted by evolution. Since the genetic search used for training does not require error derivatives, a wide range of neural models can be constructed. This generality is contrasted with various existing methods for complex system modeling, which investigate only restricted topological subsets rather than the complete class of architectures. An improvement in the predictive accuracy and parsimony of models is reported, against backpropagation networks and other well-engineered polynomial-based methods for two problems: MacKey-Glass and Lorenz-like chaotic systems. The authors also demonstrate the importance of the selection pressure towards model parsimony for the improvement of prediction accuracy.

[1]  Neal B. Abraham,et al.  Lorenz-like chaos in NH3-FIR lasers , 1995 .

[2]  Manoel Fernando Tenorio,et al.  Self-organizing network for optimum supervised learning , 1990, IEEE Trans. Neural Networks.

[3]  J. Rissanen Stochastic Complexity and Modeling , 1986 .

[4]  Schloss Birlinghoven,et al.  MDL-Based Fitness Functions for Learning Parsimonious Programs , 1995 .

[5]  Hitoshi Iba,et al.  System Identification using Structured Genetic Algorithms , 1993, ICGA.

[6]  James D. Keeler,et al.  Predicting the Future: Advantages of Semilocal Units , 1991, Neural Computation.

[7]  Heinz Mühlenbein,et al.  The Science of Breeding and Its Application to the Breeder Genetic Algorithm (BGA) , 1993, Evolutionary Computation.

[8]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[9]  Andreas S. Weigend,et al.  Time Series Prediction: Forecasting the Future and Understanding the Past , 1994 .

[10]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[11]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[12]  Raymond McLeod,et al.  Comparing undergraduate courses in systems analysis and design , 1996, CACM.

[13]  David E. Goldberg,et al.  Genetic and evolutionary algorithms come of age , 1994, CACM.

[14]  Byoung-Tak Zhang,et al.  Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor , 1993, Complex Syst..