How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues

After classically conditioned learning, dopaminergic cells in the substantia nigra pars compacta (SNc) respond immediately to unexpected conditioned stimuli (CS) but omit formerly seen responses to expected unconditioned stimuli, notably rewards. These cells play an important role in reinforcement learning. A neural model explains the key neurophysiological properties of these cells before, during, and after conditioning, as well as related anatomical and neurophysiological data about the pedunculopontine tegmental nucleus (PPTN), lateral hypothalamus, ventral striatum, and striosomes. The model proposes how two parallel learning pathways from limbic cortex to the SNc, one devoted to excitatory conditioning (through the ventral striatum, ventral pallidum, and PPTN) and the other to adaptively timed inhibitory conditioning (through the striosomes), control SNc responses. The excitatory pathway generates CS-induced excitatory SNc dopamine bursts. The inhibitory pathway prevents dopamine bursts in response to predictable reward-related signals. When expected rewards are not received, striosomal inhibition of SNc that is unopposed by excitation results in a phasic drop in dopamine cell activity. The adaptively timed inhibitory learning uses an intracellular spectrum of timed responses that is proposed to be similar to adaptively timed cellular mechanisms in the hippocampus and cerebellum. These mechanisms are proposed to include metabotropic glutamate receptor-mediated Ca2+ spikes that occur with different delays in striosomal cells. A dopaminergic burst in concert with a Ca2+ spike is proposed to potentiate inhibitory learning. The model provides a biologically predictive alternative to temporal difference conditioning models and explains substantially more data than alternative models.

[1]  G. Mogenson,et al.  Subpallidal projections to the mesencephalic locomotor region investigated with a combination of behavioral and electrophysiological recording techniques , 1986, Brain Research Bulletin.

[2]  K. Nakamura,et al.  Lateral hypothalamus neuron involvement in integration of natural and artificial rewards and cue signals. , 1986, Journal of neurophysiology.

[3]  S. Grossberg,et al.  ART 2: self-organization of stable category recognition codes for analog input patterns. , 1987, Applied optics.

[4]  D. Wilkie Stimulus intensity affects pigeons’ timing behavior: Implications for an internal clock model , 1987 .

[5]  G. Mogenson,et al.  Hippocampal signal transmission to the pedunculopontine nucleus and its regulation by dopamine D2 receptors in the nucleus accumbens: An electrophysiological and behavioural study , 1987, Neuroscience.

[6]  E. Scarnati,et al.  An EM and Golgi study on the connection between the nucleus tegmenti pedunculopontinus and the pars compacta of the substantia nigra in the rat. , 1988, Journal fur Hirnforschung.

[7]  Stephen Grossberg,et al.  Neural dynamics of adaptive timing and temporal discrimination during associative learning , 1989, Neural Networks.

[8]  P. Goldman-Rakic,et al.  Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. , 1989, Journal of neurophysiology.

[9]  H. Kita,et al.  Intracellular study of rat globus pallidus neurons: membrane properties and responses to neostriatal, subthalamic and nigral stimulation , 1991, Brain Research.

[10]  P. Calabresi,et al.  Long‐term Potentiation in the Striatum is Unmasked by Removing the Voltage‐dependent Magnesium Block of NMDA Receptor Channels , 1992, The European journal of neuroscience.

[11]  S. Grossberg,et al.  A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. , 1992, Brain research. Cognitive brain research.

[12]  H. Kita Responses of globus pallidus neurons to cortical stimulation: intracellular study in the rat , 1992, Brain Research.

[13]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[14]  C. Gerfen The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia. , 1992, Annual review of neuroscience.

[15]  A. Grace,et al.  Role of the subthalamic nucleus in the regulation of nigral dopamine neuron activity , 1992, Synapse.

[16]  P. Calabresi,et al.  Long-term synaptic depression in the striatum: physiological and pharmacological characterization , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[17]  C. Gerfen The neostriatal mosaic: multiple levels of compartmental organization , 1992, Trends in Neurosciences.

[18]  W. Schultz,et al.  Neuronal activity in monkey ventral striatum related to the expectation of reward , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[19]  H. Fibiger,et al.  Afferent connections of the laterodorsal and the pedunculopontine tegmental nuclei in the rat: A retro‐ and antero‐grade transport and immunohistochemical study , 1992, The Journal of comparative neurology.

[20]  D. S. Zahm,et al.  The patterns of afferent innervation of the core and shell in the “Accumbens” part of the rat ventral striatum: Immunohistochemical detection of retrogradely transported fluoro‐gold , 1993, The Journal of comparative neurology.

[21]  R. J. McDonald,et al.  A triple dissociation of memory systems: hippocampus, amygdala, and dorsal striatum. , 1993, Behavioral neuroscience.

[22]  D. Linden,et al.  Cellular mechanisms of long-term depression in the cerebellum , 1993, Current Opinion in Neurobiology.

[23]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  S. Greenfield,et al.  Synaptic connections between pars compacta and pars reticulata neurones: electrophysiological evidence for functional modules within the substantia nigra , 1994, Brain Research.

[25]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[26]  P. Calabresi,et al.  Post-receptor mechanisms underlying striatal long-term depression , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  W. Schultz,et al.  Importance of unpredictability for reward responses in primate dopamine neurons. , 1994, Journal of neurophysiology.

[28]  Charles J. Wilson,et al.  Surround inhibition among projection neurons is weak or nonexistent in the rat neostriatum. , 1994, Journal of neurophysiology.

[29]  Dean V. Buonomano,et al.  Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses , 1999, Neural Computation.

[30]  A. Graybiel,et al.  Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[31]  O. Hikosaka Models of information processing in the basal Ganglia edited by James C. Houk, Joel L. Davis and David G. Beiser, The MIT Press, 1995. $60.00 (400 pp) ISBN 0 262 08234 9 , 1995, Trends in Neurosciences.

[32]  JM Tepper,et al.  GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[33]  S. T. Kitai,et al.  Glutamatergic and cholinergic inputs from the pedunculopontine tegmental nucleus to dopamine neurons in the substantia nigra pars compacta , 1995, Neuroscience Research.

[34]  Colin Wilson The contribution of cortical neurons to the firing pattern of striatal spiny neurons , 1995 .

[35]  A. Dickinson,et al.  Reward-related signals carried by dopamine neurons. , 1995 .

[36]  J. Wickens,et al.  Cellular models of reinforcement. , 1995 .

[37]  J. Wickens,et al.  Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex In vitro , 1996, Neuroscience.

[38]  F. Crépel,et al.  Cellular mechanisms of long-term depression in the cerebellum , 1996 .

[39]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  S. Grossberg,et al.  The Hippocampus and Cerebellum in Adaptively Timed Learning, Recognition, and Movement , 1996, Journal of Cognitive Neuroscience.

[41]  S. Grossberg,et al.  Metabotropic Glutamate Receptor Activation in Cerebellar Purkinje Cells as Substrate for Adaptive Timing of the Classically Conditioned Eye-Blink Response , 1996, The Journal of Neuroscience.

[42]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[43]  A. Chiba,et al.  The amygdala and emotion , 1996, Current Opinion in Neurobiology.

[44]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[45]  S. T. Kitai,et al.  Two types of cholinergic neurons in the rat tegmental pedunculopontine nucleus: electrophysiological and morphological characterization , 1997, Neuroscience.

[46]  P. Calabresi,et al.  Enhancement of NMDA responses by group I metabotropic glutamate receptor activation in striatal neurones , 1997, British journal of pharmacology.

[47]  M. Inase,et al.  Excitotoxic lesions of the pedunculopontine tegmental nucleus produce contralateral hemiparkinsonism in the monkey , 1997, Neuroscience Letters.

[48]  George J. Augustine,et al.  Local calcium signalling by inositol-1,4,5-trisphosphate in Purkinje cell dendrites , 1998, Nature.

[49]  J. F. Dormont,et al.  The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat II. Effects of reversible inactivation by intracerebral microinjections , 1998, Experimental Brain Research.

[50]  W. Schultz,et al.  Learning of sequential movements by neural network model with dopamine-like reinforcement signal , 1998, Experimental Brain Research.

[51]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[52]  T. Sejnowski,et al.  A Computational Model of How the Basal Ganglia Produce Sequences , 1998, Journal of Cognitive Neuroscience.

[53]  Arthur Konnerth,et al.  A new class of synaptic response involving calcium release in dendritic spines , 1998, Nature.

[54]  L. Kaczmarek,et al.  Formation of intermediate-conductance calcium-activated potassium channels by interaction of Slack and Slo subunits , 1998, Nature Neuroscience.

[55]  H. Condé,et al.  The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat I. Context-dependent and reinforcement-related single unit activity , 1998, Experimental Brain Research.

[56]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[57]  P. Garris,et al.  Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation , 1999, Nature.

[58]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.