Self-association and Hebbian learning in linear neural networks

Studies Hebbian learning in linear neural networks with emphasis on the self-association information principle. This criterion, in one-layer networks, leads to the space of the principal components and can be generalized to arbitrary architectures. The self-association paradigm appears to be very promising because it accounts for the fundamental features of Hebbian synaptic learning and generalizes the various techniques proposed for adaptive principal component networks. The authors also include a set of simulations that compare various neural architectures and algorithms.

[1]  Kurt Hornik,et al.  Backpropagation and unsupervised learning in linear networks , 1995 .

[2]  Jie Zhu,et al.  Anti-Hebbian learning in topologically constrained linear networks: a tutorial , 1993, IEEE Trans. Neural Networks.

[3]  Pierre Baldi,et al.  Linear Learning: Landscapes and Algorithms , 1988, NIPS.

[4]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[5]  P. Foldiak,et al.  Adaptive network for optimal linear feature extraction , 1989, International 1989 Joint Conference on Neural Networks.

[6]  Nathan Intrator,et al.  Feature Extraction Using an Unsupervised Neural Network , 1992, Neural Computation.

[7]  Tomas Hrycej.  Supporting supervised learning by self-organization , 1992, Neurocomputing.

[8]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[9]  R. Brockett.  Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems , 1991 .

[10]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[11]  F. Palmieri.  A self-organizing neural network for multidimensional approximation , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[12]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[13]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[14]  F. Palmieri,et al.  Hebbian learning and self-association in nonlinear neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[15]  R. Linsker,et al.  From basic network principles to neural architecture , 1986 .

[16]  R. Vaccaro.  SVD and Signal Processing II: Algorithms, Analysis and Applications , 1991 .

[17]  R Linsker,et al.  From basic network principles to neural architecture: emergence of orientation columns. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[18]  David Zipser,et al.  Feature Discovery by Competive Learning , 1985, Cogn. Sci..

[19]  Sun-Yuan Kung,et al.  A neural network learning algorithm for adaptive principal component extraction (APEX) , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[20]  F. Palmieri.  A self-organizing neural network for nonlinear filtering , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[21]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[22]  Lei Xu,et al.  Least MSE reconstruction by self-organization. I. Multi-layer neural-nets , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[23]  Ray H. White,et al.  Competitive hebbian learning: Algorithm and demonstrations , 1992, Neural Networks.

[24]  Peter Földiák,et al.  Learning Invariance from Transformation Sequences , 1991, Neural Comput..

[25]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  Kurt Hornik,et al.  Convergence analysis of local feature extraction algorithms , 1992, Neural Networks.

[27]  Tomas Hrycej,et al.  Self-organization by delta rule , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[28]  Nathan Intrator,et al.  Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions , 1992, Neural Networks.

[29]  Tomas Hrycej,et al.  Unsupervised Learning by Backward Inhibition , 1989, IJCAI.

[30]  Francesco Palmieri,et al.  Linear neural networks which minimize the output variance , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[31]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Second Edition , 1988, Springer Series in Information Sciences.

[32]  Terence D. Sanger,et al.  Optimal unsupervised learning , 1988, Neural Networks.

[33]  Terence D. Sanger,et al.  An Optimality Principle for Unsupervised Learning , 1988, NIPS.

[34]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[35]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[36]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[37]  David J. C. MacKay,et al.  Analysis of Linsker's Simulations of Hebbian Rules , 1990, Neural Computation.

[38]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[39]  Lei Xu,et al.  Self-Organizing Rules for Robust Principal Component Analysis , 1992, NIPS.

[40]  Jun Zhang.  Dynamics and Formation of Self-Organizing Maps , 1991, Neural Computation.

[41]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[42]  S. Kung,et al.  Neural networks for extracting unsymmetric principal components , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.