Evolving Neural Network Ensembles by Fitness Sharing

The difference between evolving neural networks and evolving neural network ensembles is that the solution of evolving neural networks is an evolved neural network while the solution of evolving neural network ensemble is an evolved population of neural networks. In the practice of evolving neural network ensemble, it is common that each individual rather the whole population is evaluated. During the evolution, the solution of evolving neural networks would be better and better while it might not be the case for the solution of evolving neural network ensembles. It suggests that the final evolved population might be worse so that it is not wise to choose the final population as a solution. Through experimental studies, this paper gives ideas of how to evolve better populations.

[1]  David H. Wolpert,et al.  A Mathematical Theory of Generalization: Part II , 1990, Complex Syst..

[2]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[3]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[4]  Xin Yao,et al.  Evolving a cooperative population of neural networks by minimizing mutual information , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[5]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[6]  C. S. Wallace,et al.  Coding Decision Trees , 1993, Machine Learning.

[7]  Xin Yao,et al.  Maintaining Population Diversity By Minimizing Mutual Information , 2002, GECCO.

[8]  Xin Yao,et al.  Evolutionary Artificial Neural Networks , 1993, Int. J. Neural Syst..

[9]  Robert G. Reynolds,et al.  Evolutionary computation: Towards a new philosophy of machine intelligence , 1997 .

[10]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[11]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[12]  Xin Yao,et al.  A review of evolutionary artificial neural networks , 1993, Int. J. Intell. Syst..

[13]  Robert L. Winkler,et al.  Limits for the Precision and Value of Information from Dependent Sources , 1985, Oper. Res..

[14]  Xin Yao,et al.  Towards Designing Neural Network Ensembles by Evolution , 1998, PPSN.

[15]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[16]  H. Akaike.  A new look at the statistical model identification , 1974 .

[17]  Xin Yao,et al.  The Evolution of Connectionist Networks , 1994 .

[18]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[19]  Hussein A. Abbass,et al.  Analyzing Anti–correlation in Ensemble Learning , 2007 .

[20]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[21]  Xin Yao,et al.  Speciation as automatic categorical modularization , 1997, IEEE Trans. Evol. Comput..

[22]  Jan C. van der Lubbe,et al.  Information theory , 1997 .

[23]  Xin Yao,et al.  Learning and Evolution by Minimization of Mutual Information , 2002, PPSN.

[24]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[25]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[26]  Robert A. Jacobs,et al.  Bias/Variance Analyses of Mixtures-of-Experts Architectures , 1997, Neural Computation.

[27]  Xin Yao,et al.  Simultaneous training of negatively correlated neural networks in an ensemble , 1999, IEEE Trans. Syst. Man Cybern. Part B.