Unsupervised learningin neural computation

In this article, we consider unsupervised learningfrom the point of view of applyingneural computation on signal and data analysis problems. The article is an introductory survey, concentrating on the main principles and categories of unsupervised learning. In neural computation, there are two classical categories for unsupervised learning methods and models: ,rst, extensions of principal component analysis and factor analysis, and second, learningvector coding or clusteringmethods that are based on competitive learning . These are covered in this article. The more recent trend in unsupervised learningis to consider this problem in the framework of probabilistic generative models. If it is possible to build and estimate a model that explains the data in terms of some latent variables, key insights may be obtained into the true nature and structure of the data. This approach is also brie.y reviewed. c 2002 Elsevier Science B.V. All rights reserved.

[1]  Helge J. Ritter,et al.  Neural computation and self-organizing maps - an introduction , 1992, Computation and neural systems series.

[2]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[3]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[4]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[5]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[6]  Erkki Oja,et al.  Neural fitting: Robustness by anti-Hebbian learning , 1996, Neurocomputing.

[7]  Sun-Yuan Kung,et al.  Principal Component Neural Networks: Theory and Applications , 1996 .

[8]  Nathalie Japkowicz,et al.  Nonlinear Autoassociation Is Not Equivalent to PCA , 2000, Neural Computation.

[9]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[10]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[11]  E. Oja,et al.  Independent Component Analysis , 2013 .

[12]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[13]  Juha Karhunen,et al.  A Unified Neural Bigradient Algorithm for robust PCA and MCA , 1996, Int. J. Neural Syst..

[14]  S. Grossberg.  Neural Networks and Natural Intelligence , 1988 .

[15]  Kurt Hornik,et al.  Learning in linear neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[16]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[17]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[18]  E. B. Andersen,et al.  Modern factor analysis , 1961 .

[19]  Geoffrey E. Hinton,et al.  Unsupervised learning : foundations of neural computation , 1999 .

[20]  Antti Honkela,et al.  Bayesian Non-Linear Independent Component Analysis by Multi-Layer Perceptrons , 2000 .

[21]  T. Villmann,et al.  Topology Preservation in Self-Organizing Maps , 1999 .

[22]  Kimmo Kiviluoto,et al.  Topology preservation in self-organizing maps , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[23]  Lei Xu,et al.  Temporal BYY learning for state space approach, hidden Markov model, and blind source separation , 2000, IEEE Trans. Signal Process..

[24]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[25]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[26]  Stephen Grossberg,et al.  Direct perception or adaptive resonance? , 1980, Behavioral and Brain Sciences.

[27]  H. Harman.  Modern factor analysis , 1961 .

[28]  P. Foldiak,et al.  Adaptive network for optimal linear feature extraction , 1989, International 1989 Joint Conference on Neural Networks.

[29]  Robert J. Schalkoff,et al.  Pattern recognition - statistical, structural and neural approaches , 1991 .

[30]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[31]  H. B. Barlow,et al.  Unsupervised Learning , 1989, Neural Computation.

[32]  Geoffrey E. Hinton,et al.  Recognizing Handwritten Digits Using Mixtures of Linear Models , 1994, NIPS.

[33]  Erkki Oja,et al.  Kohonen Maps , 1999, Encyclopedia of Machine Learning.

[34]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[35]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[36]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[37]  Roman Bek,et al.  Discourse on one way in which a quantum-mechanics language on the classical logical base can be built up , 1978, Kybernetika.

[38]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[39]  Samuel Kaski,et al.  Self organization of a massive document collection , 2000, IEEE Trans. Neural Networks Learn. Syst..

[40]  Jean-Francois Cardoso,et al.  Blind signal separation: statistical principles , 1998, Proc. IEEE.

[41]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[42]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[43]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[44]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[45]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[46]  C. Malsburg.  Self-organization of orientation sensitive cells in the striate cortex , 2004, Kybernetik.

[47]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[48]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .