Evolving a cooperative population of neural networks by minimizing mutual information

Evolutionary ensembles with negative correlation learning (EENCL) is an evolutionary learning system for learning and designing neural network ensembles (Liu et al., 2000). The fitness sharing used in EENCL was based on the idea of "covering" the same training patterns by shared individuals. This paper explores connection between fitness sharing and information concept, and introduces mutual information into EENCL. Through minimization of mutual information, a diverse and cooperative population of neural networks can be evolved by EENCL. The effectiveness of such evolutionary learning approach was tested on two real-world problems.

[1]  Jan C. van der Lubbe,et al.  Information theory , 1997 .

[2]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[3]  Xin Yao,et al.  Towards Designing Neural Network Ensembles by Evolution , 1998, PPSN.

[4]  L. Goddard Information Theory , 1962, Nature.

[5]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[6]  Xin Yao,et al.  A cooperative ensemble learning system , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[7]  Robert G. Reynolds,et al.  Evolutionary computation: Towards a new philosophy of machine intelligence , 1997 .

[8]  Robert A. Jacobs,et al.  Methods For Combining Experts' Probability Assessments , 1995, Neural Computation.

[9]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[10]  Xin Yao,et al.  Making use of population information in evolutionary artificial neural networks , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[11]  M. Stone Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[12]  Xin Yao,et al.  A new evolutionary system for evolving artificial neural networks , 1997, IEEE Trans. Neural Networks.

[13]  Xin Yao,et al.  Simultaneous training of negatively correlated neural networks in an ensemble , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[16]  Robert L. Winkler,et al.  Limits for the Precision and Value of Information from Dependent Sources , 1985, Oper. Res..

[17]  X. Yao,et al.  How to Make Best Use of Evolutionary Learning , 1996 .