Methods for adaptive combination of classifiers with application to recognition of handwritten characters

Abstract.This paper discusses two techniques for improving the recognition accuracy for online handwritten character recognition: committee classification and adaptation to the user. Combining classifiers is a common method for improving recognition performance. Improvements are possible because the member classifiers may make different errors. Much variation exists in handwritten characters, and adaptation is one feasible way of dealing with such variation. Even though adaptation is usually performed for single classifiers, it is also possible to use adaptive committees. Some novel adaptive committee structures, namely, the dynamically expanding context (DEC), modified current best learning (MCBL), and class-confidence critic combination (CCCC), are presented and evaluated. They are shown to be able to improve on their member classifiers, with CCCC offering the best performance. Also, the effect of having either more or less diverse sets of member classifiers is considered.

[1]  Erkki Oja,et al.  Adaptive local subspace classifier in on-line recognition of handwritten characters , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[2]  Robert P. W. Duin,et al.  Is independence good for combining classifiers? , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[3]  Ching Y. Suen,et al.  A Method of Combining Multiple Experts for the Recognition of Unconstrained Handwritten Numerals , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[5]  Erkki Oja,et al.  Experiments with adaptation strategies for a prototype-based recognition system for isolated handwritten characters , 2001, International Journal on Document Analysis and Recognition.

[6]  Djamel Bouchaffra,et al.  A Methodology for Mapping Scores to Probabilities , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Adam Krzyżak,et al.  Methods of combining multiple classifiers and their applications to handwriting recognition , 1992, IEEE Trans. Syst. Man Cybern..

[8]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[9]  Isabelle Guyon,et al.  UNIPEN project of on-line data exchange and recognizer benchmarks , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[10]  Kari Torkkola.  An efficient way to learn English grapheme-to-phoneme rules automatically , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Sargur N. Srihari,et al.  Decision Combination in Multiple Classifier Systems , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[13]  Erkki Oja,et al.  Dynamically expanding context as committee adaptation method in on-line recognition of handwritten Latin characters , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[14]  Ahmad Fuad Rezaur Rahman,et al.  Enhancing consensus in multiple expert decision fusion , 2000 .

[15]  Erkki Oja,et al.  Application of Adaptive Committee Classifiers in On-Line Character Recognition , 2001, ICAPR.

[16]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Harris Drucker,et al.  Boosting Performance in Neural Networks , 1993, Int. J. Pattern Recognit. Artif. Intell..

[18]  Ching Y. Suen,et al.  A theoretical analysis of the application of majority voting to pattern recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[19]  Ah-Hwee Tan,et al.  Learning by supervised clustering and matching , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[20]  Chunheng Wang,et al.  Adaptive combination of classifiers and its application to handwritten Chinese character recognition , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Sung-Bae Cho,et al.  Multiple recognizers system using two-stage combination , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[22]  David J. Miller,et al.  Critic-driven ensemble classification , 1999, IEEE Trans. Signal Process..

[23]  Thomas Niesler,et al.  A variable-length category-based n-gram language model , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[24]  Shane S. Sturrock,et al.  Time Warps, String Edits, and Macromolecules – The Theory and Practice of Sequence Comparison . David Sankoff and Joseph Kruskal. ISBN 1-57586-217-4. Price £13.95 (US$22·95). , 2000 .

[25]  Seong-Whan Lee,et al.  Combining classifiers based on minimization of a Bayes error rate , 1999, Proceedings of the Fifth International Conference on Document Analysis and Recognition. ICDAR '99 (Cat. No.PR00318).

[26]  Ah-Hwee Tan,et al.  Adaptive integration of multiple experts , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[27]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[28]  Improving vocabulary independent HMM decoding results by using the dynamically expanding context , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[29]  Ahmad Fuad Rezaur Rahman,et al.  A comparative study of decision combination strategies for a novel multiple-expert classifier , 1997 .

[30]  Teuvo Kohonen,et al.  Correction of quasiphoneme strings by the dynamically expanding context , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.