Correlations strike back (again): the case of associative memory retrieval

It has long been recognised that statistical dependencies in neuronal activity need to be taken into account when decoding stimuli encoded in a neural population. Less studied, though equally pernicious, is the need to take account of dependencies between synaptic weights when decoding patterns previously encoded in an auto-associative memory. We show that activity-dependent learning generically produces such correlations, and failing to take them into account in the dynamics of memory retrieval leads to catastrophically poor recall. We derive optimal network dynamics for recall in the face of synaptic correlations caused by a range of synaptic plasticity rules. These dynamics involve well-studied circuit motifs, such as forms of feedback inhibition and experimentally observed dendritic nonlinearities. We therefore show how addressing the problem of synaptic correlations leads to a novel functional account of key biophysical features of the neural substrate.

[1]  M. Häusser,et al.  Synaptic Integration Gradients in Single Cortical Pyramidal Cell Dendrites , 2011, Neuron.

[2]  Peter Dayan,et al.  Uncertainty, phase and oscillatory hippocampal recall , 2006, NIPS.

[3]  Yali Amit,et al.  Precise Capacity Analysis in Binary Networks with Multiple Coding Level Inputs , 2010, Neural Computation.

[4]  Toshiyuki TANAKA Mean-field theory of Boltzmann machine learning , 1998 .

[5]  W. Abraham Metaplasticity: tuning synapses and networks for plasticity , 2008, Nature Reviews Neuroscience.

[6]  R. Palmer,et al.  Solution of 'Solvable model of a spin glass' , 1977 .

[7]  Andreas Engel,et al.  Improved retrieval in neural networks with external fields , 1989 .

[8]  M. Hasselmo,et al.  Acetylcholine and memory , 1993, Trends in Neurosciences.

[9]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[10]  P. Latham,et al.  Synergy, Redundancy, and Independence in Population Codes, Revisited , 2005, The Journal of Neuroscience.

[11]  A. Pouget,et al.  Neural correlations, population coding and computation , 2006, Nature Reviews Neuroscience.

[12]  Sen Song,et al.  Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits , 2005, PLoS biology.

[13]  D. Linden,et al.  The other side of the engram: experience-driven changes in neuronal intrinsic excitability , 2003, Nature Reviews Neuroscience.

[14]  Simon R. Schultz,et al.  The Ising decoder: reading out the activity of large neural ensembles , 2010, Journal of Computational Neuroscience.

[15]  Stefano Fusi,et al.  Long Memory Lifetimes Require Complex Synapses and Limited Sparseness , 2007, Frontiers Comput. Neurosci..

[16]  P. Dayan,et al.  Matching storage and recall: hippocampal spike timing–dependent plasticity and phase response curves , 2005, Nature Neuroscience.

[17]  W. Wildman,et al.  Theoretical Neuroscience , 2014 .

[18]  R. Kempter,et al.  Sparseness constrains the prolongation of memory lifetime via synaptic metaplasticity. , 2008, Cerebral cortex.

[19]  L. Abbott,et al.  Cascade Models of Synaptically Stored Memories , 2005, Neuron.

[20]  Sompolinsky,et al.  Storing infinite numbers of patterns in a spin-glass model of neural networks. , 1985, Physical review letters.

[21]  Yali Amit,et al.  Capacity analysis in multi-state synaptic models: a retrieval probability perspective , 2011, Journal of Computational Neuroscience.

[22]  Eero P. Simoncelli,et al.  Spatio-temporal correlations and visual signalling in a complete neuronal population , 2008, Nature.

[23]  Peter Dayan,et al.  Two is better than one: distinct roles for familiarity and recollection in retrieving palimpsest memories , 2011, NIPS.

[24]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[25]  Peter Dayan,et al.  Bayesian retrieval in associative memories with storage errors , 1998, IEEE Trans. Neural Networks.

[26]  D. J. C. MacKay Maximum Entropy Connections: Neural Networks , 1991 .