Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism

Dopamine (DA) depletion in the basal ganglia (BG) of Parkinson's patients gives rise to both frontal-like and implicit learning impairments. Dopaminergic medication alleviates some cognitive deficits but impairs those that depend on intact areas of the BG, apparently due to DA overdose. These findings are difficult to accommodate with verbal theories of BG/DA function, owing to complexity of system dynamics: DA dynamically modulates function in the BG, which is itself a modulatory system. This article presents a neural network model that instantiates key biological properties and provides insight into the underlying role of DA in the BG during learning and execution of cognitive tasks. Specifically, the BG modulates the execution of actions (e.g., motor responses and working memory updating) being considered in different parts of the frontal cortex. Phasic changes in DA, which occur during error feedback, dynamically modulate the BG threshold for facilitating/suppressing a cortical command in response to particular stimuli. Reduced dynamic range of DA explains Parkinson and DA overdose deficits with a single underlying dysfunction, despite overall differences in raw DA levels. Simulated Parkinsonism and medication effects provide a theoretical basis for behavioral data in probabilistic classification and reversal tasks. The model also provides novel testable predictions for neuropsychological and pharmacological studies, and motivates further investigation of BG/DA interactions with the prefrontal cortex in working memory.

[1]  M. Moscovitch,et al.  Memory for temporal order and conditional associative-learning in patients with Parkinson's disease , 1990, Neuropsychologia.

[2]  Anthony S. Maida,et al.  Using TD learning to simulate working memory performance in a model of the prefrontal cortex and basal ganglia , 2007, Cognitive Systems Research.

[3]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[4]  C. D. Stern,et al.  Handbook of Chemical Neuroanatomy Methods in Chemical Neuroanatomy. Edited by A. Bjorklund and T. Hokfelt. Elsevier, Amsterdam, 1983. Cloth bound, 548 pp. UK £140. (Volume 1 in the series). , 1986, Neurochemistry International.

[5]  J. Wickens Basal ganglia: structure and computations. , 1997 .

[6]  A. G. Smith,et al.  The Dopamine D3/D2 Receptor Agonist 7-OH-DPAT Induces Cognitive Impairment in the Marmoset , 1999, Pharmacology Biochemistry and Behavior.

[7]  Peter Redgrave,et al.  A computational model of action selection in the basal ganglia. I. A new functional anatomy , 2001, Biological Cybernetics.

[8]  T. Robbins,et al.  l-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson’s disease , 2003, Neuropsychologia.

[9]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[10]  Michael J. Frank,et al.  Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making , 2006, Neural Networks.

[11]  Christian C. Ruff,et al.  Short- and long-term changes in anterior cingulate activation during resolution of task-set competition , 2006, Brain Research.

[12]  G. E. Alexander,et al.  Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, "prefrontal" and "limbic" functions. , 1990, Progress in brain research.

[13]  A. Levey,et al.  Differential expression of D1 and D2 dopamine and m4 muscarinic acetylcholine receptor proteins in identified striatonigral neurons , 1997, Synapse.

[14]  D. Surmeier,et al.  Coordinated expression of dopamine receptors in neostriatal medium spiny neurons. , 1998, Advances in pharmacology.

[15]  J. Houk,et al.  Model of cortical-basal ganglionic processing: encoding the serial order of sensory events. , 1998, Journal of neurophysiology.

[16]  Shawn W. Ell,et al.  Category learning deficits in Parkinson's disease. , 2003, Neuropsychology.

[17]  Morris Moscovitch,et al.  Dissociating habit and recollection: evidence from Parkinson’s disease, amnesia and focal lesion patients , 2002, Neuropsychologia.

[18]  Chantal E. Stern,et al.  The nucleus accumbens in monkeys (Macaca fascicularis) , 2004, Experimental Brain Research.

[19]  Michael J. Frank,et al.  Testing Computational Models of Dopamine and Noradrenaline Dysfunction in Attention Deficit/Hyperactivity Disorder , 2007, Neuropsychopharmacology.

[20]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[21]  W. T. Maddox,et al.  Striatal contributions to category learning: Quantitative modeling of simple linear and complex nonlinear rule learning in patients with Parkinson's disease , 2001, Journal of the International Neuropsychological Society.

[22]  J. Mink THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS , 1996, Progress in Neurobiology.

[23]  G E Alexander,et al.  Movement-related neuronal activity selectively coding either direction or muscle pattern in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[24]  G. Percheron,et al.  Parallel processing in the basal ganglia: up to a point , 1991, Trends in Neurosciences.

[25]  R. O’Reilly,et al.  Separate neural substrates for skill learning and performance in the ventral and dorsal striatum , 2007, Nature Neuroscience.

[26]  C. Gerfen,et al.  D1 and D2 dopamine receptor function in the striatum: coactivation of D1- and D2-dopamine receptors on separate populations of neurons results in potentiated immediate early gene response in D1-containing neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[27]  G. E. Alexander,et al.  Preparation for movement: neural representations of intended direction in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[28]  Michael J. Frank,et al.  A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. , 2006, Behavioral neuroscience.

[29]  M. Naoi,et al.  The metabolism of l-DOPA and l-threo-3,4-dihydroxyphenylserine and their effects on monoamines in the human brain: analysis of the intraventricular fluid from parkinsonian patients , 1996, Journal of the Neurological Sciences.

[30]  M. Gluck,et al.  Probabilistic classification learning in amnesia. , 1994, Learning & memory.

[31]  P. Goldman-Rakic,et al.  Dopamine D2 receptor mechanisms contribute to age-related cognitive decline: the effects of quinpirole on memory and motor performance in monkeys , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[32]  Paul M. Grasby,et al.  A positron emission tomography (PET) investigation of the role of striatal dopamine (D2) receptor availability in spatial cognition , 2005, NeuroImage.

[33]  Trevor W. Robbins,et al.  Enhanced and Impaired Attentional Performance After Infusion of D1 Dopaminergic Receptor Agents into Rat Prefrontal Cortex , 2000, The Journal of Neuroscience.

[34]  S. Foote,et al.  Extrathalamic modulation of cortical function. , 1987, Annual review of neuroscience.

[35]  Y Agid,et al.  Are dopaminergic neurons selectively vulnerable to Parkinson's disease? , 1993, Advances in neurology.

[36]  O Hikosaka,et al.  Neural systems for control of voluntary action--a hypothesis. , 1998, Advances in biophysics.

[37]  P. Calabresi,et al.  Abnormal Synaptic Plasticity in the Striatum of Mice Lacking Dopamine D2 Receptors , 1997, The Journal of Neuroscience.

[38]  K. Jellinger,et al.  Recent developments in the pathology of Parkinson's disease. , 2002, Journal of neural transmission. Supplementum.

[39]  P. Strick,et al.  Basal Ganglia Output and Cognition: Evidence from Anatomical, Behavioral, and Clinical Studies , 2000, Brain and Cognition.

[40]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[41]  M. Frank,et al.  Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. , 2006, Psychological review.

[42]  L. Tremblay,et al.  Abnormal spontaneous activity of globus pallidus neurons in monkeys with MPTP-induced parkinsonism , 1991, Brain Research.

[43]  P. O’Donnell Dopamine gating of forebrain neural ensembles , 2003, The European journal of neuroscience.

[44]  L. Henderson,et al.  Serial reaction time learning and Parkinson's disease: Evidence for a procedural learning deficit , 1995, Neuropsychologia.

[45]  W. Schultz,et al.  Neuronal activity in monkey striatum related to the expectation of predictable environmental events. , 1992, Journal of neurophysiology.

[46]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[47]  K. Doya Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[48]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[49]  Patricia M. Berretty,et al.  On the dominance of unidimensional rules in unsupervised categorization , 1999, Perception & psychophysics.

[50]  C. Gerfen The neostriatal mosaic: multiple levels of compartmental organization in the basal ganglia. , 1992, Annual review of neuroscience.

[51]  Okihide Hikosaka,et al.  Role of basal ganglia in initiation of voluntary movements , 1988 .

[52]  Elsevier Biomedical Press RESPONSES OF STRIATAL NEURONS IN THE BEHAVING MONKEY. 1. HEAD OF THE CAUDATE NUCLEUS , 1983 .

[53]  R. O’Reilly Six principles for biologically based computational models of cortical cognition , 1998, Trends in Cognitive Sciences.

[54]  Y. Agid,et al.  Delayed response tasks in basal ganglia lesions in man: Further evidence for a striato-frontal cooperation in behavioural adaptation , 1996, Neuropsychologia.

[55]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[56]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[57]  P. Greengard,et al.  Bidirectional Regulation of DARPP-32 Phosphorylation by Dopamine , 1997, The Journal of Neuroscience.

[58]  John G. Taylor,et al.  A neural model of working memory processes in normal subjects, Parkinson's disease and schizophrenia for fMRI design and predictions , 2000, Neural Networks.

[59]  Charles J. Wilson,et al.  The origins of two-state spontaneous membrane potential fluctuations of neostriatal spiny neurons , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[60]  C. Marsden,et al.  'Frontal' cognitive function in patients with Parkinson's disease 'on' and 'off' levodopa. , 1988, Brain : a journal of neurology.

[61]  M. Sugimori,et al.  Excitatory nature of dopamine in the nigro-caudate pathway , 1976, Experimental Brain Research.

[62]  Michael J. Frank,et al.  When Memory Fails, Intuition Reigns , 2006, Psychological science.

[63]  J. Penney,et al.  The functional anatomy of basal ganglia disorders , 1989, Trends in Neurosciences.

[64]  T. Robbins,et al.  Differential effects of 6-OHDA lesions of the frontal cortex and caudate nucleus on the ability to acquire an attentional set. , 2001, Cerebral cortex.

[65]  Michael J. Frank,et al.  Transitivity, flexibility, conjunctive representations, and the hippocampus. II. A computational analysis , 2003, Hippocampus.

[66]  A. Pickering,et al.  The New Reinforcement Sensitivity Theory: Implications for Personality Measurement , 2006, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[67]  Thomas Rammsayer,et al.  Sensorimotor effects of pergolide, a dopamine agonist, in healthy subjects: a lateralized readiness potential study , 2006, Psychopharmacology.

[68]  C. Gerfen,et al.  Neostriatal dopamine receptors , 1994, Trends in Neurosciences.

[69]  J. Horvitz,et al.  Opposing Roles of D1 and D2 Receptors in Appetitive Conditioning , 2003, The Journal of Neuroscience.

[70]  J. Bargas,et al.  D 1 Receptor Activation Enhances Evoked Discharge in Neostriatal Medium Spiny Neurons by Modulating an L-Type Ca 2 1 Conductance , 1997 .

[71]  T. Robbins,et al.  Probabilistic learning and reversal deficits in patients with Parkinson’s disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication , 2000, Neuropsychologia.

[72]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[73]  P. Salin,et al.  Bilateral 6‐Hydroxydopamine‐induced Lesion of the Nigrostriatal Dopamine Pathway Reproduces the Effects of Unilateral Lesion on Substance P but not on Enkephalin Expression in Rat Basal Ganglia , 1996, The European journal of neuroscience.

[74]  Carol A. Seger,et al.  Striatal activation during acquisition of a cognitive skill. , 1999, Neuropsychology.

[75]  Hans Forssberg,et al.  Anatomical and physiological evidence for D1 and D2 dopamine receptor colocalization in neostriatal neurons , 2000, Nature Neuroscience.

[76]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[77]  Michael Van Elzakker,et al.  Transitivity, flexibility, conjunctive representations, and the hippocampus. I. An empirical analysis , 2003, Hippocampus.

[78]  Corey J. Bohil,et al.  Delayed feedback effects on rule-based and information-integration category learning. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[79]  Avishai Henik,et al.  Disinhibition of Automatic Word Reading in Parkinson's Disease , 1993, Cortex.

[80]  A. Nieoullon Dopamine and the regulation of cognition and attention , 2002, Progress in Neurobiology.

[81]  Neill R. Taylor,et al.  Modelling the frontal lobes in health and disease , 1999 .

[82]  R. Cools Dopaminergic modulation of cognitive function-implications for l-DOPA treatment in Parkinson's disease , 2006, Neuroscience & Biobehavioral Reviews.

[83]  H. Allain,et al.  Procedural Memory in Recent-Onset Parkinson’s Disease , 1999, Dementia and Geriatric Cognitive Disorders.

[84]  C. Gerfen Molecular effects of dopamine on striatal-projection pathways , 2000, Trends in Neurosciences.

[85]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[86]  P. Goldman-Rakic Regional and cellular fractionation of working memory. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[87]  T. Robbins,et al.  Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[88]  M. Gluck,et al.  How do people solve the "weather prediction" task?: individual variability in strategies for probabilistic category learning. , 2002, Learning & memory.

[89]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[90]  Peter Redgrave,et al.  Basal Ganglia , 2020, Encyclopedia of Autism Spectrum Disorders.

[91]  S. Kish,et al.  Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease. Pathophysiologic and clinical implications. , 1988, The New England journal of medicine.

[92]  M. Reuter,et al.  The influence of the dopaminergic system on cognitive functioning: A molecular genetic approach , 2005, Behavioural Brain Research.

[93]  Charles J. Wilson,et al.  The generation of natural firing patterns in neostriatal neurons. , 1993, Progress in brain research.

[94]  M. Farah,et al.  Effects of bromocriptine on human subjects depend on working memory capacity , 1997, Neuroreport.

[95]  T. Robbins,et al.  Dissociating executive mechanisms of task control following frontal lobe damage and Parkinson's disease. , 1998, Brain : a journal of neurology.

[96]  Corey J Bohil,et al.  Observational versus feedback training in rule-based and information-integration category learning , 2002, Memory & cognition.

[97]  G Maura,et al.  Release-regulating D-2 dopamine receptors are located on striatal glutamatergic nerve terminals. , 1988, The Journal of pharmacology and experimental therapeutics.

[98]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[99]  Paul Greengard,et al.  Loss of bidirectional striatal synaptic plasticity in L-DOPA–induced dyskinesia , 2003, Nature Neuroscience.

[100]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[101]  M. Petrides,et al.  Wisconsin Card Sorting Revisited: Distinct Neural Circuits Participating in Different Stages of the Task Identified by Event-Related Functional Magnetic Resonance Imaging , 2001, The Journal of Neuroscience.

[102]  G. E. Alexander,et al.  Functional architecture of basal ganglia circuits: neural substrates of parallel processing , 1990, Trends in Neurosciences.

[103]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[104]  C. Marsden,et al.  Learning manual pursuit tracking skills in patients with Parkinson's disease. , 1997, Brain : a journal of neurology.

[105]  D. Finch,et al.  Plasticity of responses to synaptic inputs in rat ventral striatal neurons after repeated administration of the dopamine D2 antagonist raclopride , 1999, Synapse.

[106]  J. O'Doherty,et al.  Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex , 2003, Science.

[107]  L R Squire,et al.  Intact learning of artificial grammars and intact category learning by patients with Parkinson's disease. , 1999, Behavioral neuroscience.

[108]  P. Calabresi,et al.  Dopaminergic control of synaptic plasticity in the dorsal striatum , 2001, The European journal of neuroscience.

[109]  P. Goldman-Rakic,et al.  Dopamine D1 receptor mechanisms in the cognitive performance of young adult and aged monkeys , 1994, Psychopharmacology.

[110]  T. Robbins,et al.  Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. , 2001, Cerebral cortex.

[111]  T. Sejnowski,et al.  Dopamine-mediated stabilization of delay-period activity in a network model of prefrontal cortex. , 2000, Journal of neurophysiology.

[112]  M. Bear,et al.  Synaptic plasticity: LTP and LTD , 1994, Current Opinion in Neurobiology.

[113]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[114]  D. Bub,et al.  Task switching deficits associated with Parkinson’s disease reflect depleted attentional resources , 2002, Neuropsychologia.

[115]  P. Greengard,et al.  Modulation of calcium currents by a D1 dopaminergic protein kinase/phosphatase cascade in rat neostriatal neurons , 1995, Neuron.

[116]  E. T. Rolls,et al.  Responses of striatal neurons in the behaving monkey. 3. Effects of iontophoretically applied dopamine on normal responsiveness , 1984, Neuroscience.

[117]  Yuko Munakata,et al.  When Actions Speak Louder Than Words , 2006, Psychological science.

[118]  B. Dubois,et al.  [Cognitive functions and the basal ganglia: the model of Parkinson disease]. , 1994, Revue neurologique.

[119]  Stephen Grossberg,et al.  How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades , 2004, Neural Networks.

[120]  J. Cohen,et al.  Context, cortex, and dopamine: a connectionist approach to behavior and biology in schizophrenia. , 1992, Psychological review.

[121]  J. Bargas,et al.  D2 Dopamine Receptors in Striatal Medium Spiny Neurons Reduce L-Type Ca2+ Currents and Excitability via a Novel PLCβ1–IP3–Calcineurin-Signaling Cascade , 2000, The Journal of Neuroscience.

[122]  M. Gluck,et al.  Interactive memory systems in the human brain , 2001, Nature.

[123]  R. Malenka,et al.  Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. , 2000, Annual review of neuroscience.

[124]  B. Knowlton,et al.  Learning and memory functions of the Basal Ganglia. , 2002, Annual review of neuroscience.

[125]  H. Fibiger,et al.  D1 and D2 dopamine receptors differentially regulate c-fos expression in striatonigral and striatopallidal neurons , 1992, Neuroscience.

[126]  J. Deniau,et al.  Disinhibition as a basic process in the expression of striatal functions , 1990, Trends in Neurosciences.

[127]  J. Wickens,et al.  Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. , 2001, Journal of neurophysiology.

[128]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[129]  Jonathan D. Cohen,et al.  Computational perspectives on dopamine function in prefrontal cortex , 2002, Current Opinion in Neurobiology.

[130]  T. Sejnowski,et al.  The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. , 1994, Learning & memory.

[131]  Antonio R. Damasio,et al.  The Prefrontal Cortex: Its Structure, Function and Pathology , 1992 .