Feature Extraction Using an Unsupervised Neural Network

A novel unsupervised neural network for dimensionality reduction that seeks directions emphasizing multimodality is presented, and its connection to exploratory projection pursuit methods is discussed. This leads to a new statistical insight into the synaptic modification equations governing learning in Bienenstock, Cooper, and Munro (BCM) neurons (1982). The importance of a dimensionality reduction principle based solely on distinguishing features is demonstrated using a phoneme recognition experiment. The extracted features are compared with features extracted using a backpropagation network.

[1]  E. Zwicker,et al.  Subdivision of the audible frequency range into critical bands , 1961 .

[2]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[3]  K. Abromeit.  Music Received , 2023, Notes.

[4]  J. Kruskal.  TOWARD A PRACTICAL METHOD WHICH HELPS UNCOVER THE STRUCTURE OF A SET OF MULTIVARIATE OBSERVATIONS BY FINDING THE LINEAR TRANSFORMATION WHICH OPTIMIZES A NEW “INDEX OF CONDENSATION” , 1969 .

[5]  Raymond M. Wright,et al.  Numerical classification applied to certain Jamaican eocene nummulitids , 1971 .

[6]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[7]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[8]  T. Sejnowski,et al.  Storing covariance with nonlinearly interacting neurons , 1977, Journal of mathematical biology.

[9]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[10]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[11]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[12]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  J. Friedman,et al.  PROJECTION PURSUIT DENSITY ESTIMATION , 1984 .

[14]  D. Freedman,et al.  Asymptotics of Graphical Projection Pursuit , 1984 .

[15]  Leon N. Cooper,et al.  Development and properties of neural networks , 1985 .

[16]  E. Clothiaux,et al.  Modeling and simulation III: Simulation of a model for development of visual cortical specificity , 1986 .

[17]  Raymond D. Rimey,et al.  Real-Time 3-D Object Classification Using a Learning System , 1987, Other Conferences.

[18]  L. Cooper,et al.  A physiological basis for a theory of synapse modification. , 1987, Science.

[19]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[20]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[21]  Philip Lieberman,et al.  Speech Physiology, Speech Perception, and Acoustic Phonetics , 1988 .

[22]  L N Cooper,et al.  Mean-field theory of a neural network. , 1988, Proceedings of the National Academy of Sciences of the United States of America.

[23]  P. Hall.  Estimating the direction in which a data set is most interesting , 1988 .

[24]  L. Cooper,et al.  Molecular Mechanisms for Synaptic Modification in the Visual Cortex: Interaction between Theory and Experiment , 1989 .

[25]  P. Hall.  On Polynomial-Based Projection Indices for Exploratory Projection Pursuit , 1989 .

[26]  Nathan Intrator,et al.  A Neural Network for Feature Extraction , 1989, NIPS.

[27]  D. P. Morgan,et al.  A keyword spotter which incorporates neural networks for secondary processing , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[28]  B. Seebach.  Evidence for the development of phonetic property detectors in a neural net without innate knowledge of linguistic structure , 1990 .

[29]  Geoffrey E. Hinton,et al.  The Bootstrap Widrow-Hoff Rule as a Cluster-Formation Algorithm , 1990, Neural Computation.

[30]  Richard P. Lippmann,et al.  Review of Neural Networks for Speech Recognition , 1989, Neural Computation.

[31]  Nathan Intrator,et al.  Supervised and unsupervised feature extraction from a cochlear model for speech recognition , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[32]  L. Cooper,et al.  Synaptic plasticity in visual cortex: comparison of theory with experiment. , 1991, Journal of neurophysiology.

[33]  Nathan Intrator.  Feature Extraction using an Unsupervised Neural Network , 1991 .

[34]  Nathan Intrator,et al.  Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions , 1992, Neural Networks.

[35]  N. Intrator.  On the combination of supervised and unsupervised learning , 1993 .

[36]  Nathan Intrator,et al.  Three-Dimensional Object Recognition Using an Unsupervised BCM Network: The Usefulness of Distinguishing Features , 1993, Neural Computation.

[37]  Nathan Intrator,et al.  Combining Exploratory Projection Pursuit and Projection Pursuit Regression with Application to Neural Networks , 1993, Neural Computation.