A Comparative Study of Neural Network Algorithms Applied to Optical Character Recognition

Three simple general purpose networks are tested for pattern classification on an optical character recognition problem. The feed-forward (multi-layer perceptron) network, the Hopfield network and a competitive learning network are compared. The input patterns are obtained by optically scanning images of printed digits and uppercase letters. The resulting data is used as input for the networks with two-state input nodes; for others, features are extracted by template matching and pixel counting. The classification capabilities of the networks are compared with a nearest neighbour algorithm applied to the same feature vectors. The feed-forward network reaches the same recognition rates as the nearest neighbour algorithm, even when only a small percentage of the possible connections is used. The Hopfield network performs less well, and overloading of the network remains a problem. Recognition rates with the competitive learning network, if input patterns are clustered well, are again as high as the nearest neighbour algorithm.

[1]  D. J. Wallace,et al.  Learning and memory properties in fully connected networks , 1987 .

[2]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[3]  Richard J. Sasiela.  Forgetting as a way to improve neural-net behavior , 1987 .

[4]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[5]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[6]  Thomas M. Thompson.  From error-correcting codes through sphere packings to simple groups , 1983 .

[7]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[8]  Oscar Firschein,et al.  An approach to general pattern recognition , 1962, IRE Trans. Inf. Theory.

[9]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[10]  J. J. Hopfield,et al.  ‘Unlearning’ has a stabilizing effect in collective memories , 1983, Nature.

[11]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[12]  Yann LeCun,et al.  Une procedure d'apprentissage pour reseau a seuil asymmetrique (A learning scheme for asymmetric threshold networks) , 1985 .

[13]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[14]  W. Highleyman.  Linear Decision Functions, with Application to Pattern Recognition , 1962, Proceedings of the IRE.

[15]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[16]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[17]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .