Modeling the manifolds of images of handwritten digits

This paper describes two new methods for modeling the manifolds of digitized images of handwritten digits. The models allow a priori information about the structure of the manifolds to be combined with empirical data. Accurate modeling of the manifolds allows digits to be discriminated using the relative probability densities under the alternative models. One of the methods is grounded in principal components analysis, the other in factor analysis. Both methods are based on locally linear low-dimensional approximations to the underlying data manifold. Links with other methods that model the manifold are discussed.

[1]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[2]  Bernard Widrow,et al.  The "Rubber-Mask" Technique I. Pattern Measurement and Analysis , 1973 .

[3]  Bernard Widrow,et al.  The "rubber-mask" technique - I. Pattern measurement and analysis , 1973, Pattern Recognit..

[4]  Martin A. Fischler,et al.  The Representation and Matching of Pictorial Structures , 1973, IEEE Transactions on Computers.

[5]  J. Fleiss.  Statistical methods for rates and proportions , 1974 .

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[8]  Brian Everitt,et al.  An Introduction to Latent Variable Models , 1984 .

[9]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[10]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[11]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[12]  Yann LeCun,et al.  Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Network , 1991, NIPS.

[13]  Yann LeCun,et al.  Efficient Pattern Recognition Using a New Transformation Distance , 1992, NIPS.

[14]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[15]  Todd K. Leen,et al.  Fast nonlinear dimension reduction , 1993, IEEE International Conference on Neural Networks.

[16]  Nanda Kambhatla,et al.  Fast Non-Linear Dimension Reduction , 1993, NIPS.

[17]  Patrice Y. Simard.  Efficient Computation of Complex Distance Metrics Using Hierarchical Filtering , 1993, NIPS.

[18]  Maurice Milgram,et al.  Transformation Invariant Autoassociation with Application to Handwritten Character Recognition , 1994, NIPS.

[19]  Patrice Y. Simard,et al.  Learning Prototype Models for Tangent Distance , 1994, NIPS.

[20]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Stephen M. Omohundro,et al.  Nonlinear Image Interpolation using Manifold Learning , 1994, NIPS.

[22]  Geoffrey E. Hinton,et al.  Recognizing Handwritten Digits Using Mixtures of Linear Models , 1994, NIPS.

[23]  R. Zemel.  A minimum description length framework for unsupervised learning , 1994 .

[24]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[25]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[26]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[27]  Holger Schwenk,et al.  Learning Discriminant Tangent Models for Handwritten Character Recognition , 1995 .

[28]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Geoffrey E. Hinton,et al.  Using Generative Models for Handwritten Digit Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Peter Dayan,et al.  Factor Analysis Using Delta-Rule Wake-Sleep Learning , 1997, Neural Computation.

[31]  Tomaso A. Poggio,et al.  Example-Based Learning for View-Based Human Face Detection , 1998, IEEE Trans. Pattern Anal. Mach. Intell..