A Memetic Pareto Evolutionary Approach to Artificial Neural Networks

Evolutionary Artificial Neural Networks (EANN) have been a focus of research in the areas of Evolutionary Algorithms (EA) and Artificial Neural Networks (ANN) for the last decade. In this paper, we present an EANN approach based on pareto multi-objective optimization and differential evolution augmented with local search. We call the approach Memetic Pareto Artificial Neural Networks (MPANN). We show empirically that MPANN is capable to overcome the slow training of traditional EANN with equivalent or better generalization.

[1]  R. Storn,et al.  Differential evolution a simple and efficient adaptive scheme for global optimization over continu , 1997 .

[2]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[3]  Hussein A. Abbass,et al.  Solving Multiobjective Optimization Problems using Evolutionary Algorithm , 2001 .

[4]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[5]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[6]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[7]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[8]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[9]  Xin Yao,et al.  Evolutionary Artificial Neural Networks , 1993, Int. J. Neural Syst..

[10]  Hussein A. Abbass,et al.  Solving Two Multi-Objective Optimization Problems Using Evolutionary Algorithm , 2003 .

[11]  Wei Yan,et al.  A hybrid genetic/BP algorithm and its application for radar target classification , 1997, Proceedings of the IEEE 1997 National Aerospace and Electronics Conference. NAECON 1997.

[12]  PoliRiccardo,et al.  Evolving the Topology and the Weights of Neural Networks Using a Dual Representation , 1998 .

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[15]  C. A. Coello Coello,et al.  A Comprehensive Survey of Evolutionary-Based Multiobjective Optimization Techniques , 1999, Knowledge and Information Systems.

[16]  David E. Goldberg,et al.  A niched Pareto genetic algorithm for multiobjective optimization , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[17]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[18]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[19]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[20]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[21]  Huan Liu,et al.  Book review: Machine Learning, Neural and Statistical Classification Edited by D. Michie, D.J. Spiegelhalter and C.C. Taylor (Ellis Horwood Limited, 1994) , 1996, SGAR.

[22]  D B Fogel,et al.  Evolving neural networks for detecting breast cancer. , 1995, Cancer letters.

[23]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[24]  Xin Yao,et al.  Towards designing artificial neural networks by evolution , 1998 .

[25]  James F. Frenzel,et al.  Training product unit neural networks with genetic algorithms , 1993, IEEE Expert.

[26]  E C Wasson,et al.  A step toward computer-assisted mammography using evolutionary programming and neural networks. , 1997, Cancer letters.

[27]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[28]  David B. Fogel,et al.  Alternative Neural Network Training Methods , 1995, IEEE Expert.

[29]  Xin Yao,et al.  Making use of population information in evolutionary artificial neural networks , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[30]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[31]  Pablo Moscato,et al.  Memetic algorithms: a short introduction , 1999 .

[32]  Robert G. Reynolds,et al.  Evolutionary computation: Towards a new philosophy of machine intelligence , 1997 .

[33]  H. Abbass,et al.  PDE: a Pareto-frontier differential evolution approach for multi-objective optimization problems , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).