Selective Learning for Multilayer Feedforward Neural Networks

Selective learning is an active learning strategy where the neural network selects during training the most informative patterns. This paper investigates a selective learning strategy where the informativeness of a pattern is measured as the sensitivity of the network output to perturbations in that pattern. The sensitivity approach to selective learning is then compared with an error selection approach where pattern informativeness is defined as the approximation error.

[1]  Andries P. Engelbrecht,et al.  Selective learning using sensitivity analysis , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[2]  Andries Petrus Engelbrecht,et al.  Incremental learning using sensitivity analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[3]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[4]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[5]  Gregory J. Wolff,et al.  Optimal Brain Surgeon: Extensions and performance comparisons , 1993, NIPS 1993.

[6]  Byoung-Tak Zhang,et al.  Accelerated Learning by Active Example Selection , 1994, Int. J. Neural Syst..

[7]  David E. Rumelhart,et al.  Generalization by Weight-Elimination with Application to Forecasting , 1990, NIPS.

[8]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[9]  Yoshio Hirose,et al.  Backpropagation algorithm which varies the number of hidden units , 1991, International 1989 Joint Conference on Neural Networks.

[10]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[11]  Jacek M. Zurada,et al.  Perturbation method for deleting redundant inputs of perceptron networks , 1997, Neurocomputing.

[12]  John R. Deller,et al.  Selective training of feedforward artificial neural networks using matrix perturbation theory , 1995, Neural Networks.

[13]  A. R. Technische,et al.  The Dynamic Pattern Selection Algorithm: Eeective Training and Controlled Generalization of Backpropagation Neural Networks , 1994 .

[14]  Mark Plutowski,et al.  Selecting concise training sets from clean data , 1993, IEEE Trans. Neural Networks.

[15]  Partha Niyogi,et al.  A Formulation for Active Learning with Applications to Object Detection , 1995 .