A Computational Model of How the Basal Ganglia Produce Sequences

We propose a systems-level computational model of the basal ganglia based closely on known anatomy and physiology. First, we assume that the thalamic targets, which relay ascending information to cortical action and planning areas, are tonically inhibited by the basal ganglia. Second, we assume that the output stage of the basal ganglia, the internal segment of the globus pallidus (GPi), selects a single action from several competing actions via lateral interactions. Third, we propose that a form of local working memory exists in the form of reciprocal connections between the external globus pallidus (GPe) and the subthalamic nucleus (STN). As a test of the model, the system was trained to learn a sequence of states that required the context of previous actions. The striatum, which was assumed to represent a conjunction of cortical states, directly selected the action in the GP during training. The STN-to-GP connection strengths were modified by an associative learning rule and came to encode the sequence after 20 to 40 iterations through the sequence. Subsequently, the system automatically reproduced the sequence when cued to the first action. The behavior of the model was found to be sensitive to the ratio of the striatal-nigral learning rate to the STN-GP learning rate. Additionally, the degree of striatal inhibition of the globus pallidus had a significant influence on both learning and the ability to select an action. Low learning rates, which would be hypothesized to reflect low levels of dopamine, as in Parkinson's disease, led to slow acquisition of contextual information. However, this could be partially offset by modeling a lesion of the globus pallidus that resulted in an increase in the gain of the STN units. The parameter sensitivity of the model is discussed within the framework of existing behavioral and lesion data.

[1]  Irwin J. Kopin,et al.  The Biochemical Basis of Neuropharmacology , 1971, The Yale Journal of Biology and Medicine.

[2]  J. Fuster Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. , 1973, Journal of neurophysiology.

[3]  W. Precht The synaptic organization of the brain G.M. Shepherd, Oxford University Press (1975). 364 pp., £3.80 (paperback) , 1976, Neuroscience.

[4]  Kenneth W. Flowers,et al.  Lack of prediction in the motor behaviour of Parkinsonism. , 1978, Brain : a journal of neurology.

[5]  M. Horstink,et al.  Cognitive and motor shifting aptitude disorder in Parkinson's disease. , 1984, Journal of neurology, neurosurgery, and psychiatry.

[6]  Charles J. Wilson Postsynaptic potentials evoked in spiny neostriatal projection neurons by stimulation of ipsilateral and contralateral neocortex , 1986, Brain Research.

[7]  P. Goldman-Rakic,et al.  Topography of Corticostriatal Projections in Nonhuman Primates and Implications for Functional Parcellation of the Neostriatum , 1986 .

[8]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[9]  N. Swerdlow,et al.  Dopamine, schizophrenia, mania, and depression: Toward a unified hypothesis of cortico-striatopallido-thalamic function , 1987, Behavioral and Brain Sciences.

[10]  Stephen Wolfram,et al.  Mathematica: a system for doing mathematics by computer (2nd ed.) , 1991 .

[11]  C. Marsden,et al.  Internal versus external cues and the control of attention in Parkinson's disease. , 1988, Brain : a journal of neurology.

[12]  M. Nissen,et al.  On the development of procedural knowledge. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[13]  C. Wilson,et al.  Intracellular recording of identified neostriatal patch and matrix spiny cells in a slice preparation preserving cortical inputs. , 1989, Journal of neurophysiology.

[14]  A. Parent Extrinsic connections of the basal ganglia , 1990, Trends in Neurosciences.

[15]  G. E. Alexander,et al.  Functional architecture of basal ganglia circuits: neural substrates of parallel processing , 1990, Trends in Neurosciences.

[16]  A. Graybiel Neurotransmitters and neuromodulators in the basal ganglia , 1990, Trends in Neurosciences.

[17]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[18]  J. Hedreen,et al.  Organization of striatopallidal, striatonigral, and nigrostriatal projections in the macaque , 1991, The Journal of comparative neurology.

[19]  James L. McClelland,et al.  Learning the structure of event sequences. , 1991, Journal of experimental psychology. General.

[20]  W. T. Thach,et al.  Basal ganglia motor control. II. Late pallidal timing relative to movement onset and inconsistent pallidal coding of movement parameters. , 1991, Journal of neurophysiology.

[21]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[22]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[23]  P. Calabresi,et al.  Long-term synaptic depression in the striatum: physiological and pharmacological characterization , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Peter C. Jurs,et al.  Mathematica , 2019, J. Chem. Inf. Comput. Sci..

[25]  W. Schultz,et al.  Neuronal activity in monkey ventral striatum related to the expectation of reward , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  K. Clark,et al.  Auto- and cross-correlation analysis of subthalamic nucleus neuronal activity in neostriatal- and globus pallidal-lesioned rats , 1992, Brain Research.

[27]  P. Strick,et al.  Multiple output channels in the basal ganglia. , 1993, Science.

[28]  H J Sagar,et al.  Alternating fluency in Parkinson's disease. An evaluation of the attentional control theory of cognitive impairment. , 1993, Brain : a journal of neurology.

[29]  Terrence J. Sejnowski,et al.  Foraging in an Uncertain Environment Using Predictive Hebbian Learning , 1993, NIPS.

[30]  T. Robbins,et al.  Contrasting mechanisms of impaired attentional set-shifting in patients with frontal lobe damage or Parkinson's disease. , 1993, Brain : a journal of neurology.

[31]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[32]  J. Fuster Frontal lobes , 1993, Current Opinion in Neurobiology.

[33]  M. Hallett,et al.  Procedural learning in Parkinson's disease and cerebellar degeneration , 1993, Annals of neurology.

[34]  L. Ryan,et al.  Subthalamic nucleus lesion regularizes firing patterns in globus pallidus and substantia nigra pars reticulata neurons in rats , 1993, Brain Research.

[35]  M R DeLong,et al.  The primate subthalamic nucleus. III. Changes in motor behavior and neuronal activity in the internal pallidum induced by subthalamic inactivation in the MPTP model of parkinsonism. , 1994, Journal of neurophysiology.

[36]  H. Bergman,et al.  The primate subthalamic nucleus. I. Functional properties in intact animals. , 1994, Journal of neurophysiology.

[37]  C. Marsden,et al.  The behavioural and motor consequences of focal lesions of the basal ganglia in man. , 1994, Brain : a journal of neurology.

[38]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[39]  R. Llinás,et al.  Electrophysiology of globus pallidus neurons in vitro. , 1994, Journal of neurophysiology.

[40]  G. Barrionuevo,et al.  Excitatory stimulation during postsynaptic inhibition induces long-term depression in hippocampus in vivo. , 1994, Journal of neurophysiology.

[41]  R. Malenka,et al.  Simultaneous LTP of non-NMDA- and LTD of NMDA-receptor-mediated responses in the nucleus accumbens , 1994, Nature.

[42]  X. D. Yang,et al.  Weak excitation and simultaneous inhibition induce long-term depression in hippocampal CA1 neurons. , 1994, Journal of neurophysiology.

[43]  T. Wichmann,et al.  Synaptic innervation of neurons in the internal pallidal segment by the subthalamic nucleus and the external pallidum in monkeys , 1994, The Journal of comparative neurology.

[44]  A. Flaherty,et al.  Input-output organization of the sensorimotor striatum in the squirrel monkey , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[45]  T. Sejnowski,et al.  The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. , 1994, Learning & memory.

[46]  P. Strick,et al.  Anatomical evidence for cerebellar and basal ganglia involvement in higher cognitive function. , 1994, Science.

[47]  JW Polli,et al.  Expression of a calmodulin-dependent phosphodiesterase isoform (PDE1B1) correlates with brain regions having extensive dopaminergic innervation , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[48]  A M Graybiel,et al.  The basal ganglia and adaptive motor control. , 1994, Science.

[49]  T. Wichmann,et al.  The External Pallidum and the Subthalamic Nucleus Send Convergent Synaptic Inputs onto Single Neurones in the Internal Pallidal Segment in Monkey:Anatomical Organization and Functional Significance , 1994 .

[50]  C. Marsden,et al.  The functions of the basal ganglia and the paradox of stereotaxic surgery in Parkinson's disease. , 1994, Brain : a journal of neurology.

[51]  S. Cabib,et al.  Opposite responses of mesolimbic dopamine system to controllable and uncontrollable aversive experiences , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[52]  B. Alger,et al.  GABAergic and developmental influences on homosynaptic LTD and depotentiation in rat hippocampus , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[53]  A. Barto Adaptive Critics and the Basal Ganglia , 1995 .

[54]  R. Lonser,et al.  The results, indications, and physiology of posteroventral pallidotomy for patients with Parkinson's disease. , 1995, Neurosurgery.

[55]  Peter Dayan,et al.  Bee foraging in uncertain environments using predictive hebbian learning , 1995, Nature.

[56]  H Mushiake,et al.  Pallidal neuron activity during sequential arm movements. , 1995, Journal of neurophysiology.

[57]  J. Wickens,et al.  Cellular models of reinforcement. , 1995 .

[58]  T. Sejnowski,et al.  How the Basal Ganglia Make Decisions , 1996 .

[59]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  K. Berridge,et al.  Implementation of Action Sequences by a Neostriatal Site: A Lesion Mapping Study of Grooming Syntax , 1996, The Journal of Neuroscience.

[61]  Garrett E. Alexander Basal ganglia , 1998 .

[62]  M. Ebert,et al.  The Biochemical Basis of Neuropharmacology, 7th ed , 1998 .

[63]  J. Caston,et al.  Effects of electrolytic lesions of the lateral pallidum on motor coordination, spatial learning, and regional brain variations of cytochrome oxidase activity in rats , 1999, Behavioural Brain Research.

[64]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[65]  Tomoki Fukai,et al.  Sequence generation in arbitrary temporal patterns from theta-nested gamma oscillations: a model of the basal ganglia-thalamo-cortical loops , 1999, Neural Networks.

[66]  K. Doya,et al.  Parallel neural networks for learning sequential procedures , 1999, Trends in Neurosciences.