Reverse replay of behavioural sequences in hippocampal place cells during the awake state

The hippocampus has long been known to be involved in spatial navigational learning in rodents, and in memory for events in rodents, primates and humans. A unifying property of both navigation and event memory is a requirement for dealing with temporally sequenced information. Reactivation of temporally sequenced memories for previous behavioural experiences has been reported in sleep in rats. Here we report that sequential replay occurs in the rat hippocampus during awake periods immediately after spatial experience. This replay has a unique form, in which recent episodes of spatial experience are replayed in a temporally reversed order. This replay is suggestive of a role in the evaluation of event sequences in the manner of reinforcement learning models. We propose that such replay might constitute a general mechanism of learning and memory.

[1]  W. Scoville,et al.  LOSS OF RECENT MEMORY AFTER BILATERAL HIPPOCAMPAL LESIONS , 1957, Journal of neurology, neurosurgery, and psychiatry.

[2]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[3]  J. B. Ranck,et al.  Studies on single neurons in dorsal hippocampal formation and septum in unrestrained rats. I. Behavioral correlates and firing repertoires. , 1973, Experimental neurology.

[4]  D. Olton,et al.  Animal Behavior Processes , 2022 .

[5]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[6]  R. Passingham.  The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[7]  R. Morris,et al.  Place navigation impaired in rats with hippocampal lesions , 1982, Nature.

[8]  G. Buzsáki.  Hippocampal sharp waves: Their origin and significance , 1986, Brain Research.

[9]  G. Buzsáki.  Two-stage model of memory trace formation: A role for “noisy” brain states , 1989, Neuroscience.

[10]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[11]  William H. Press,et al.  Numerical recipes in C++: the art of scientific computing, 2nd Edition (C++ ed., print. is corrected to software version 2.10) , 1994 .

[12]  B L McNaughton,et al.  Dynamics of the hippocampal ensemble code for space. , 1993, Science.

[13]  D. Gaffan.  Scene-Specific Memory for Objects: A Model of Episodic Memory Impairment in Monkeys with Fornix Transection , 1994, Journal of Cognitive Neuroscience.

[14]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[15]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[16]  M. Wilson,et al.  Coordinated Interactions between Hippocampal Ripples and Cortical Spindles during Slow-Wave Sleep , 1998, Neuron.

[17]  R. Morris,et al.  Delay‐dependent impairment of a matching‐to‐place task with chronic and intrahippocampal infusion of the NMDA‐antagonist D‐AP5 , 1999, Hippocampus.

[18]  Pawel Cichosz,et al.  An Analysis of Experience Replay in Temporal Difference Learning , 1999, Cybern. Syst..

[19]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[20]  J. Hobson,et al.  Visual Discrimination Task Improvement: A Multi-Step Process Occurring During Sleep , 2000, Journal of Cognitive Neuroscience.

[21]  M. Wilson,et al.  Temporally Structured Replay of Awake Hippocampal Ensemble Activity during Rapid Eye Movement Sleep , 2001, Neuron.

[22]  Albert K. Lee,et al.  Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep , 2002, Neuron.

[23]  W. Press,et al.  Numerical Recipes in C++: The Art of Scientific Computing (2nd edn)1 Numerical Recipes Example Book (C++) (2nd edn)2 Numerical Recipes Multi-Language Code CD ROM with LINUX or UNIX Single-Screen License Revised Version3 , 2003 .

[24]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[25]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.