DIVACE: Diverse and Accurate Ensemble Learning Algorithm

In order for a neural network ensemble to generalise properly, two factors are considered vital. One is the diversity and the other is the accuracy of the networks that comprise the ensemble. There exists a tradeoff as to what should be the optimal measures of diversity and accuracy. The aim of this paper is to address this issue. We propose the DIVACE algorithm which tries to produce an ensemble as it searches for the optimum point on the diversity-accuracy curve. The DIVACE algorithm formulates the ensemble learning problem as a multi-objective problem explicitly.

[1]  Hussein A. Abbass.  Pareto Neuro-Ensembles , 2003, Australian Conference on Artificial Intelligence.

[2]  Xin Yao,et al.  Evolving Neural Network Ensembles by Minimization of Mutual Information , 2004, Int. J. Hybrid Intell. Syst..

[3]  R. Storn,et al.  Differential evolution a simple and efficient adaptive scheme for global optimization over continu , 1997 .

[4]  Hussein A. Abbass,et al.  Pareto neuro-evolution: constructing ensemble of neural networks using multi-objective optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[5]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[6]  Kalyanmoy Deb,et al.  MULTI-OBJECTIVE FUNCTION OPTIMIZATION USING NON-DOMINATED SORTING GENETIC ALGORITHMS , 1994 .

[7]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[8]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[9]  Xin Yao,et al.  Diversity creation methods: a survey and categorisation , 2004, Inf. Fusion.

[10]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[11]  Xin Yao,et al.  Learning and Evolution by Minimization of Mutual Information , 2002, PPSN.

[12]  H. Abbass,et al.  PDE: a Pareto-frontier differential evolution approach for multi-objective optimization problems , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[13]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[14]  Hussein A. Abbass,et al.  A Memetic Pareto Evolutionary Approach to Artificial Neural Networks , 2001, Australian Joint Conference on Artificial Intelligence.