Formal Neural Networks: From Supervised to Unsupervised Learning

This lecture is on the study of formal neural networks. The emphasis will be put on the bridges that exists between the analysis of the main tasks and architectures that are usually considered: auto-associative learning by an attractor neural network, hetero-associative learning by a feedforward net, learning a rule by example and unsupervised learning. In particular a duality between two architectures will be shown to provide a tool for comparing supervised and unsupervised learning.

[1]  E. Gardner The space of interactions in neural network models , 1988 .

[2]  Mario Marchand,et al.  Learning by Minimizing Resources in Neural Networks , 1989, Complex Syst..

[3]  E. Gardner,et al.  Zero temperature parallel dynamics for infinite range spin glasses and neural networks , 1987 .

[4]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[5]  Michael Biehl,et al.  The AdaTron: An Adaptive Perceptron Algorithm , 1989 .

[6]  L. F. Abbott Learning in neural network memories , 1990 .

[7]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[8]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[9]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[10]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[11]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[12]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[13]  Jean-Pierre Nadal,et al.  Neural trees: a new tool for classification , 1990 .

[14]  Haim Sompolinsky,et al.  STATISTICAL MECHANICS OF NEURAL NETWORKS , 1988 .

[15]  Néstor Parga,et al.  Information processing by a perceptron in an unsupervised learning task , 1993 .

[16]  D. O. Hebb,et al.  The organization of behavior , 1988 .

[17]  Naftali Tishby,et al.  Consistent inference of probabilities in layered networks: predictions and generalizations , 1989, International 1989 Joint Conference on Neural Networks.

[18]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[19]  Vladimir Vapnik,et al.  Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics) , 1982 .

[20]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[21]  M. Golea,et al.  A Growth Algorithm for Neural Network Decision Trees , 1990 .

[22]  J. Nadal,et al.  Information capacity of a perceptron , 1992 .

[23]  Personnaz,et al.  Collective computational properties of neural networks: New learning mechanisms. , 1986, Physical review. A, General physics.

[24]  Néstor Parga,et al.  Duality Between Learning Machines: A Bridge Between Supervised and Unsupervised Learning , 1994, Neural Computation.

[25]  Peter Seitz,et al.  Minimum class entropy: A maximum information approach to layered networks , 1989, Neural Networks.

[26]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[27]  P. Peretto An introduction to the modeling of neural networks , 1992 .

[28]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[29]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[30]  E. Gardner Maximum Storage Capacity in Neural Networks , 1987 .

[31]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[32]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Richard E. Blahut,et al.  Principles and practice of information theory , 1987 .