Content-addressability and learning in neural networks

The content-addressability of patterns stored in Ising-spin neural network models with symmetric interactions is studied. Numerical results from simulations on the ICL distributed array processor (DAP) involving systems with up to 2048 neurons are presented. Behaviour consistent with finite-size scaling, characteristic of a first-order phase transition, is shown to be exhibited by the basins of attraction of the stored patterns both in the case of the Hopfield model and for systems using a local iterative learning algorithm designed to optimise the basins of attraction. Estimates are obtained for the critical minimum overlaps which an input pattern must have with a stored pattern in order to successfully retrieve it.

[1]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[2]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[4]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[5]  Marc Mézard,et al.  Solvable models of working memories , 1986 .

[6]  K. Schilling,et al.  Lattice Gauge Theory: A Challenge in Large-Scale Computing , 1986 .

[7]  E. Gardner.  Structure of metastable states in the Hopfield model , 1986 .

[8]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[9]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[10]  E. Gardner.  Maximum Storage Capacity in Neural Networks , 1987 .

[11]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[12]  D. J. Wallace,et al.  Learning and memory properties in fully connected networks , 1987 .

[13]  S. Venkatesh.  Epsilon capacity of neural networks , 1987 .

[14]  D. J. Wallace,et al.  Dynamics and statistical mechanics of the Hopfield model , 1987 .

[15]  E. Gardner.  The space of interactions in neural network models , 1988 .