A global geometric framework for nonlinear dimensionality reduction.

Scientists working with large volumes of high-dimensional data, such as global climate patterns, stellar spectra, or human gene distributions, regularly confront the problem of dimensionality reduction: finding meaningful low-dimensional structures hidden in their high-dimensional observations. The human brain confronts the same problem in everyday perception, extracting from its high-dimensional sensory inputs-30,000 auditory nerve fibers or 10(6) optic nerve fibers-a manageably small number of perceptually relevant features. Here we describe an approach to solving dimensionality reduction problems that uses easily measured local metric information to learn the underlying global geometry of a data set. Unlike classical techniques such as principal component analysis (PCA) and multidimensional scaling (MDS), our approach is capable of discovering the nonlinear degrees of freedom that underlie complex natural observations, such as human handwriting or images of a face under different viewing conditions. In contrast to previous algorithms for nonlinear dimensionality reduction, ours efficiently computes a globally optimal solution, and, for an important class of data manifolds, is guaranteed to converge asymptotically to the true structure.

[1]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[2]  W. Klein,et al.  Vowel spectra, vowel spaces, and vowel identification. , 1970, The Journal of the Acoustical Society of America.

[3]  R. Shepard,et al.  Perceptual illusion of rotation of three-dimensional objects. , 1976, Science.

[4]  P. Menozzi,et al.  Synthetic maps of human gene frequencies in Europeans. , 1978, Science.

[5]  R N Shepard,et al.  Multidimensional Scaling, Tree-Fitting, and Clustering , 1980, Science.

[6]  Richard Durbin,et al.  An analogue approach to the travelling salesman problem using an elastic net method , 1987, Nature.

[7]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[8]  D Zipser,et al.  Learning the hidden structure of speech. , 1988, The Journal of the Acoustical Society of America.

[9]  J. Freyd,et al.  Apparent Motion of the Human Body , 1990 .

[10]  F A Mussa-Ivaldi,et al.  Computations underlying the execution of movement: a biological perspective. , 1991, Science.

[11]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[12]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[13]  B J Richmond,et al.  Concurrent processing and complexity of temporally encoded neuronal messages in visual perception. , 1991, Science.

[14]  M. Young,et al.  Sparse population coding of faces in the inferotemporal cortex. , 1992, Science.

[15]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[16]  George Karypis,et al.  Introduction to Parallel Computing , 1994 .

[17]  R. Shepard Perceptual-cognitive universals as reflections of the world , 1994, Psychonomic bulletin & review.

[18]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[19]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[20]  J. Hurrell Decadal Trends in the North Atlantic Oscillation: Regional Temperatures and Precipitation , 1995, Science.

[21]  R Hecht-Nielsen,et al.  Replicator neural networks for universal optimal source coding. , 1995, Science.

[22]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[23]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[24]  Ted von Hippel,et al.  Automated classification of stellar spectra - II. Two-dimensional classification with neural networks and principal components analysis , 1998, astro-ph/9803050.

[25]  J. Ashby References and Notes , 1999 .