Information capacity of associative memories

Associative memory networks consisting of highly interconnected binary-valued cells have been used to model neural networks. Tight asymptotic bounds have been found for the information capacity of these networks. The authors derive the asymptotic information capacity of these networks using results from normal approximation theory and theorems about exchangeable random variables. >

[1]  B. De Finetti,et al.  Funzione caratteristica di un fenomeno aleatorio , 1929 .

[2]  Lawrence D. Jackel,et al.  VLSI implementation of a neural network memory with several hundreds of neurons , 1987 .

[3]  Santosh S. Venkatesh,et al.  The capacity of the Hopfield associative memory , 1987, IEEE Trans. Inf. Theory.

[4]  H. O. Lancaster.  The Structure of Bivariate Distributions , 1958 .

[5]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[6]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[7]  W. Little.  The existence of persistent states in the brain , 1974 .

[8]  Yaser S. Abu-Mostafa,et al.  Information capacity of the Hopfield model , 1985, IEEE Trans. Inf. Theory.

[9]  W. A. Little,et al.  Analytic study of the memory storage capacity of a neural network , 1978 .

[10]  S. Venkatesh.  Epsilon capacity of neural networks , 1987 .

[11]  H. C. LONGUET-HIGGINS,et al.  Non-Holographic Associative Memory , 1969, Nature.

[12]  John H. Holland,et al.  Tests on a cell assembly theory of the action of the brain, using a large digital computer , 1956, IRE Trans. Inf. Theory.

[13]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[14]  C. Esseen.  Fourier analysis of distribution functions. A mathematical study of the Laplace-Gaussian law , 1945 .

[15]  Janos Galambos,et al.  A general Poisson limit theorem of probability theory , 1973 .

[16]  Kaoru Nakano,et al.  Associatron-A Model of Associative Memory , 1972, IEEE Trans. Syst. Man Cybern..

[17]  A. C. Berry.  The accuracy of the Gaussian approximation to the sum of independent variates , 1941 .

[18]  S. Grossberg.  How does a brain build a cognitive code , 1980 .

[19]  L. J. Savage,et al.  Symmetric measures on Cartesian products , 1955 .