Supervised Training Using an Unsupervised Approach to Active Learning

Active learning algorithms allow neural networks to dynamically take part in the selection of the most informative training patterns. This paper introduces a new approach to active learning, which combines an unsupervised clustering of training data with a pattern selection approach based on sensitivity analysis. Training data is clustered into groups of similar patterns based on Euclidean distance, and the most informative pattern from each cluster is selected for training using the sensitivity analysis incremental learning algorithm in (Engelbrecht and Cloete, 1999). Experimental results show that the clustering approach improves on standard active learning as presented in (Engelbrecht and Cloete, 1999).

[1]  Waibel.  A novel objective function for improved phoneme recognition using time delay neural networks , 1989 .

[2]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[3]  D. Mackay,et al.  Bayesian methods for adaptive models , 1992 .

[4]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[5]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[6]  Kenji Fukumizu,et al.  Active Learning in Multilayer Perceptrons , 1995, NIPS.

[7]  Mark Plutowski,et al.  Selecting concise training sets from clean data , 1993, IEEE Trans. Neural Networks.

[8]  Byoung-Tak Zhang,et al.  Accelerated Learning by Active Example Selection , 1994, Int. J. Neural Syst..

[9]  John R. Deller,et al.  Selective training of feedforward artificial neural networks using matrix perturbation theory , 1995, Neural Networks.

[10]  David A. Cohn,et al.  Neural Network Exploration Using Optimal Experiment Design , 1993, NIPS.

[11]  Jenq-Neng Hwang,et al.  Query-based learning applied to partially trained multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[12]  Andries Petrus Engelbrecht,et al.  Incremental learning using sensitivity analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[13]  Axel Röbel.  Dynamic pattern selection for faster learning and controlled generalization of neural networks , 1994, ESANN.

[14]  Andries P. Engelbrecht,et al.  Selective learning using sensitivity analysis , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[15]  Partha Niyogi,et al.  A Formulation for Active Learning with Applications to Object Detection , 1995 .

[16]  Axel Röbel.  Dynamic Pattern Selection: Effectively training Backpropagation Neural Networks , 1994 .

[17]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.