Neural Network Model of Memory Retrieval

Human memory can store large amount of information. Nevertheless, recalling is often a challenging task. In a classical free recall paradigm, where participants are asked to repeat a briefly presented list of words, people make mistakes for lists as short as 5 words. We present a model for memory retrieval based on a Hopfield neural network where transition between items are determined by similarities in their long-term memory representations. Meanfield analysis of the model reveals stable states of the network corresponding (1) to single memory representations and (2) intersection between memory representations. We show that oscillating feedback inhibition in the presence of noise induces transitions between these states triggering the retrieval of different memories. The network dynamics qualitatively predicts the distribution of time intervals required to recall new memory items observed in experiments. It shows that items having larger number of neurons in their representation are statistically easier to recall and reveals possible bottlenecks in our ability of retrieving memories. Overall, we propose a neural network model of information retrieval broadly compatible with experimental observations and is consistent with our recent graphical model (Romani et al., 2013).

[1]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[2]  Marc W. Howard,et al.  A distributed representation of temporal context , 2002 .

[3]  Bruno B Averbeck,et al.  Parallel processing of serial movements in prefrontal cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Joshua I. Sanders,et al.  Cortical interneurons that specialize in disinhibitory control , 2013, Nature.

[5]  M. Kahana.  The Cognitive Correlates of Human Brain Oscillations , 2006, The Journal of Neuroscience.

[6]  J. Wixted,et al.  Analyzing the dynamics of free recall: An integrative review of the empirical literature , 1994, Psychonomic bulletin & review.

[7]  Apostolos P. Georgopoulos,et al.  I. Single cells encode shape, sequence, and metric parameters , 2003 .

[8]  Apostolos P. Georgopoulos,et al.  Neural activity in prefrontal cortex during copying geometrical shapes , 2003, Experimental Brain Research.

[9]  B. Murdock,et al.  The immediate retention of unrelated words. , 1960, Journal of experimental psychology.

[10]  R. Shiffrin,et al.  Search of associative memory. , 1981 .

[11]  Anders Lansner,et al.  Reactivation in Working Memory: An Attractor Network Model of Free Recall , 2013, PloS one.

[12]  Stephen Grossberg,et al.  STORE working memory networks for storage and recall of arbitrary temporal sequences , 1994, Biological Cybernetics.

[13]  Daniel J. Amit,et al.  Mean Field and Capacity in Realistic Networks of Spiking Neurons Storing Sparsely Coded Random Memories , 2004, Neural Computation.

[14]  Simon Farrell,et al.  Modelling transposition latencies: Constraints for theories of serial order memory☆ , 2004 .

[15]  Richard M. Shiffrin,et al.  SAM: A theory of probabilistic search in associative memory. , 1980 .

[16]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[17]  R. Oostenveld,et al.  Theta and Gamma Oscillations Predict Encoding and Retrieval of Declarative Memory , 2006, The Journal of Neuroscience.

[18]  Invariant Object Identification.  A Neural Network Model of , 2010 .

[19]  Gordon D. A. Brown,et al.  A temporal ratio model of memory. , 2007, Psychological review.

[20]  M. Tsodyks.  ASSOCIATIVE MEMORY IN NEURAL NETWORKS WITH THE HEBBIAN LEARNING RULE , 1989 .

[21]  M. Kahana.  Associative retrieval processes in free recall , 1996, Memory & cognition.

[22]  E. Ruppin,et al.  Recall and Recognition in an Attractor Neural Network Model of Memory Retrieval , 1991 .

[23]  R. Okada,et al.  Interresponse times in single-trial free recall. , 1970 .

[24]  M. Hasselmo,et al.  Free recall and recognition in a network model of the hippocampus: simulating effects of scopolamine on human memory function , 1997, Behavioural Brain Research.

[25]  A D Baddeley,et al.  Short-term Memory for Word Sequences as a Function of Acoustic, Semantic and Formal Similarity , 1966, The Quarterly journal of experimental psychology.

[26]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[27]  Daniel J. Amit,et al.  Conversion of Temporal Correlations Between Stimuli to Spatial Correlations Between Attractors , 1999, Neural Computation.

[28]  J. Pine,et al.  Chunking mechanisms in human learning , 2001, Trends in Cognitive Sciences.

[29]  Marc W Howard,et al.  Contextual variability and serial position effects in free recall. , 1999, Journal of experimental psychology. Learning, memory, and cognition.

[30]  M. Usher,et al.  The demise of short-term memory revisited: empirical and computational investigations of recency effects. , 2005, Psychological review.

[31]  I. Fried,et al.  Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall , 2008, Science.

[32]  Sandro Romani,et al.  Scaling Laws of Associative Memory Retrieval , 2013, Neural Computation.

[33]  Geoffrey E. Hinton,et al.  Distributed Representations , 1986, The Philosophy of Artificial Intelligence.

[34]  D. Amit,et al.  Quantitative study of attractor neural networks retrieving at low spike rates: II. Low-rate retrieval in symmetric networks , 1991 .

[35]  H. Markram,et al.  Interneurons of the neocortical inhibitory system , 2004, Nature Reviews Neuroscience.

[36]  George Mandler,et al.  Effects of organization and semantic similarity on recall and recognition , 1969 .

[37]  Murdock,et al.  The serial position effect of free recall , 1962 .

[38]  S. Grossberg,et al.  Laminar cortical dynamics of cognitive and motor working memory, sequence learning and performance: toward a unified theory of how the cerebral cortex works. , 2008, Psychological review.

[39]  Marc W Howard,et al.  When Does Semantic Similarity Help Episodic Retrieval , 2002 .

[40]  Kanter,et al.  Temporal association in asymmetric neural networks. , 1986, Physical review letters.

[41]  Murdock Bb.  The immediate retention of unrelated words. , 1960 .

[42]  Pentti Kanerva,et al.  Sparse Distributed Memory , 1988 .

[43]  Per B Sederberg,et al.  The temporal contiguity effect predicts episodic memory performance. , 2010, Memory & cognition.

[44]  A. Glenberg,et al.  A temporal distinctiveness theory of recency and modality effects. , 1986, Journal of experimental psychology. Learning, memory, and cognition.

[45]  E. Tulving.  Subjective organization and effects of repetition in multi-trial free-recall learning , 1966 .

[46]  Sean M. Polyn,et al.  A context maintenance and retrieval model of organizational processes in free recall. , 2009, Psychological review.

[47]  O. Jensen,et al.  Frontal theta activity in humans increases with memory load in a working memory task , 2002, The European journal of neuroscience.

[48]  J. Wixted,et al.  An analysis of latency and interresponse time in free recall , 1994, Memory & cognition.

[49]  J. Karhu,et al.  Theta oscillations index human hippocampal activation during a working memory task. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[50]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[51]  Sandro Romani,et al.  Effects of long-term representations on free recall of unrelated words , 2015, Learning & memory.

[52]  Sandro Romani,et al.  Word length effect in free recall of randomly assembled word lists , 2014, Front. Comput. Neurosci..

[53]  D Sherrington,et al.  A neural network model of working memory exhibiting primacy and recency , 1991 .

[54]  Bard Ermentrout,et al.  A model for complex sequence learning and reproduction in neural populations , 2011, Journal of Computational Neuroscience.

[55]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[56]  S. Raghavachari,et al.  Gating of Human Theta Oscillations by a Working Memory Task , 2001, The Journal of Neuroscience.

[57]  S Grossberg,et al.  Neural dynamics of word recognition and recall: attentional priming, learning, and resonance. , 1986, Psychological review.