A neural network solution to the transverse patterning problem depends on repetition of the input code

Abstract. Using computer simulations, this paper investigates how input codes affect a minimal computational model of the hippocampal region CA3. Because encoding context seems to be a function of the hippocampus, we have studied problems that require learning context for their solution. Here we study a hippocampally dependent, configural learning problem called transverse patterning. Previously, we showed that the network does not produce long local context codings when the sequential input patterns are orthogonal, and it fails to solve many context-dependent problems in such situations. Here we show that this need not be the case if we assume that the input changes more slowly than a processing interval. Stuttering, i.e., repeating inputs, allows the network to create long local context firings even for orthogonal inputs. With these long local context firings, the network is able to solve the transverse patterning problem. Without stuttering, transverse patterning is not learned. Because stuttering is so useful, we investigate the relationship between the stuttering repetition length and relative context length in a simple, idealized sequence prediction problem. The relative context length, defined as the average length of the local context codes divided by the stuttering length, interacts with activity levels and has an optimal stuttering repetition length. Moreover, the increase in average context length can reach this maximum without loss of relative capacity. Finally, we note that stuttering is an example of maintained or introduced redundancy that can improve neural computations.

[1]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[2]  W. Levy,et al.  The relationship of local context codes to sequence length memory capacity. , 1996, Network.

[3]  W. Levy,et al.  Temporal contiguity requirements for long-term associative potentiation/depression in the hippocampus , 1983, Neuroscience.

[4]  K. I. Blum,et al.  Functional significance of long-term potentiation for sequence learning and prediction. , 1996, Cerebral cortex.

[5]  W. Levy,et al.  Insights into associative long-term potentiation from computational models of NMDA receptor-mediated calcium influx and intracellular calcium concentration changes. , 1990, Journal of neurophysiology.

[6]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[7]  R. Hirsh.  The hippocampus and contextual retrieval of information from memory: a theory. , 1974, Behavioral biology.

[8]  William B. Levy,et al.  Predicting Complex Behavior in Sparse Asymmetric Networks , 1992, NIPS.

[9]  J W Rudy,et al.  Rats with damage to the hippocampal-formation are impaired on the transverse-patterning problem but not on elemental discriminations. , 1995, Behavioral neuroscience.

[10]  J. Gray,et al.  Précis of The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system , 1982, Behavioral and Brain Sciences.

[11]  Stephen Grossberg,et al.  Studies of mind and brain , 1982 .

[12]  James L. McClelland,et al.  Hippocampal conjunctive encoding, storage, and recall: Avoiding a trade‐off , 1994, Hippocampus.

[13]  Daniel J. Amit,et al.  Modeling brain function: the world of attractor neural networks, 1st Edition , 1989 .

[14]  William B. Levy,et al.  A special role for input codes in solving the transverse patterning problem , 1997 .

[15]  R Linsker,et al.  Perceptual neural organization: some approaches based on network models and information theory. , 1990, Annual review of neuroscience.

[16]  L. Barsalou.  Cognitive Psychology: An Overview for Cognitive Scientists , 1992 .

[17]  W. Levy.  A computational approach to hippocampal function , 1989 .

[18]  John G. Taylor,et al.  Storing temporal sequences , 1991, Neural Networks.

[19]  L. Nadel,et al.  Context and conditioning: A place for space , 1980 .

[20]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[21]  J. W. Rudy,et al.  Some properties of configural learning: An investigation of the transverse-patterning problem. , 1992 .

[22]  R. Kesner,et al.  Long-term memory for contextual attributes: Dissociation of amygdala and hippocampus , 1983, Behavioural Brain Research.

[23]  William B. Levy,et al.  Setting the Activity Level in Sparse Random Networks , 1994, Neural Computation.