Mathematical foundations of neurocomputing

An attempt is made to establish a mathematical theory that shows the intrinsic mechanisms, capabilities, and limitations of information processing by various architectures of neural networks. A method of statistically analyzing one-layer neural networks is given, covering the stability of associative mapping and mapping by totally random networks. A fundamental problem of statistical neurodynamics is considered in a way that is different from the spin-glass approach. A dynamic analysis of associative memory models and a general theory of neural learning, in which the learning potential function plays a role, are given. An advanced theory of learning and self-organization is proposed, covering backpropagation and its generalizations as well as the formation of topological maps and neural representations of information. >

[1]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[2]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..

[3]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[4]  E M Harth,et al.  Brain functions and neural dynamics. , 1970, Journal of theoretical biology.

[5]  Shun-ichi Amari,et al.  Characteristics of randomly connected threshold-element networks and network systems , 1971 .

[6]  S. Amari,et al.  Characteristics of Random Nets of Analog Neuron-Like Elements , 1972, IEEE Trans. Syst. Man Cybern..

[7]  Shun-ichi Amari,et al.  Learning Patterns and Pattern Sequences by Self-Organizing Nets of Threshold Elements , 1972, IEEE Transactions on Computers.

[8]  James A. Anderson,et al.  A simple neural network generating an interactive memory , 1972 .

[9]  Kaoru Nakano,et al.  Associatron-A Model of Associative Memory , 1972, IEEE Trans. Syst. Man Cybern..

[10]  Teuvo Kohonen,et al.  Correlation Matrix Memories , 1972, IEEE Transactions on Computers.

[11]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[12]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[13]  S. Amari,et al.  A Mathematical Foundation for Statistical Neurodynamics , 1977 .

[14]  S. Geman.  SOME AVERAGING AND STABILITY RESULTS FOR RANDOM DIFFERENTIAL EQUATIONS , 1979 .

[15]  S. Amari,et al.  Existence and stability of local excitations in homogeneous neural fields , 1979, Journal of mathematical biology.

[16]  Shun-Ichi Amari,et al.  Topographic organization of nerve fields , 1979, Neuroscience Letters.

[17]  F. Tanaka,et al.  Analytic theory of the ground state properties of a spin glass. II. XY spin glass , 1980 .

[18]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[19]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[20]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  S. Amari,et al.  Competition and Cooperation in Neural Nets , 1982 .

[22]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[23]  Shun-ichi Amari,et al.  Field theory of self-organizing neural nets , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[24]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Gérard Weisbuch,et al.  Scaling laws for the attractors of Hopfield networks , 1985 .

[26]  Eric Goles Ch.,et al.  Decreasing energy functions as a tool for studying threshold networks , 1985, Discret. Appl. Math..

[27]  S. Amari,et al.  Sensori-motor transformations in the brain (with a critique of the tensor theory of cerebellum). , 1985, Journal of theoretical biology.

[28]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[29]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[30]  W. Kinzel.  Learning and pattern recognition in spin glass models , 1985 .

[31]  R. Linsker,et al.  From basic network principles to neural architecture , 1986 .

[32]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[33]  E. Gardner.  Structure of metastable states in the Hopfield model , 1986 .

[34]  R Linsker,et al.  From basic network principles to neural architecture: emergence of spatial-opponent cells. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Meir,et al.  Exact solution of a layered neural network model. , 1987, Physical review letters.

[36]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.

[37]  S. Amari,et al.  On the Topological Representation of Signals in Self-Organizing Nerve Fields , 1987 .

[38]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[39]  Terrence J. Sejnowski,et al.  Parallel Networks that Learn to Pronounce English Text , 1987, Complex Syst..

[40]  H Sompolinsky,et al.  Associative neural network model for the generation of temporal patterns. Theory and application to central pattern generators. , 1988, Biophysical journal.

[41]  S. Amari.  Dynamical stability of formation of cortical maps , 1988 .

[42]  E. Gardner.  The space of interactions in neural network models , 1988 .

[43]  Shun-ichi Amari,et al.  Statistical neurodynamics of associative memory , 1988, Neural Networks.

[44]  Shun-ichi Amari.  Associative Memory and Its Statistical Neurodynamical Analysis , 1988 .

[45]  Haim Sompolinsky,et al.  STATISTICAL MECHANICS OF NEURAL NETWORKS , 1988 .

[46]  Shun-ichi Amari,et al.  Characteristics of sparsely encoded associative memory , 1989, Neural Networks.

[47]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[48]  Barak A. Pearlmutter.  Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[49]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[50]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[51]  Kenji Doya,et al.  Adaptive neural oscillator using continuous-time back-propagation learning , 1989, Neural Networks.