Training with noise and the storage of correlated patterns in a neural network model

Local iterative learning algorithms for the interactions between Ising spins in neural network models are discussed. They converge to solutions with basins of attraction whose shape is determined by the noise in the training data, provided such solutions exist. The training is applied both to the storage of random patterns and to a model for the storage of correlated words. The existence of correlations increases the storage capacity of a given network beyond that for random patterns. The model can be modified to store cycles of patterns and in particular is applied to the storage of continuous items of English text.

[1]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[2]  I. Guyon,et al.  Information storage and retrieval in spin-glass like neural networks , 1985 .

[3]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[4]  Sompolinsky,et al.  Spin-glass models of neural networks. , 1985, Physical review. A, General physics.

[5]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[6]  Néstor Parga,et al.  The ultrametric organization of memories in a neural network , 1986 .

[7]  Personnaz,et al.  Collective computational properties of neural networks: New learning mechanisms. , 1986, Physical review. A, General physics.

[8]  E. Gardner.  Structure of metastable states in the Hopfield model , 1986 .

[9]  Baldi,et al.  Number of stable points for spin-glasses and neural networks of higher orders. , 1987, Physical review letters.

[10]  D. Amit,et al.  Statistical mechanics of neural networks near saturation , 1987 .

[11]  W. Krauth,et al.  Learning algorithms with optimal stability in neural networks , 1987 .

[12]  E. Gardner.  Multiconnected neural network models , 1987 .

[13]  E. Gardner.  Maximum Storage Capacity in Neural Networks , 1987 .

[14]  Corinna Cortes,et al.  Hierarchical associative networks , 1987 .

[15]  Opper,et al.  Learning of correlated patterns in spin-glass networks by local learning rules. , 1987, Physical review letters.

[16]  D. J. Wallace,et al.  Learning and memory properties in fully connected networks , 1987 .

[17]  S. Venkatesh.  Epsilon capacity of neural networks , 1987 .

[18]  D. J. Wallace,et al.  Dynamics and statistical mechanics of the Hopfield model , 1987 .

[19]  P. Leath,et al.  The failure distribution in percolation models of breakdown , 1987 .

[20]  Kanter,et al.  Associative recall of memory without errors. , 1987, Physical review. A, General physics.

[21]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[22]  Gutfreund.  Neural networks with hierarchically correlated patterns. , 1988, Physical review. A, General physics.

[23]  E. Gardner.  The space of interactions in neural network models , 1988 .

[24]  B. Forrest.  Content-addressability and learning in neural networks , 1988 .

[25]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.