A Neurocomputational Model of Dopamine and Prefrontal–Striatal Interactions during Multicue Category Learning by Parkinson Patients

Most existing models of dopamine and learning in Parkinson disease (PD) focus on simulating the role of basal ganglia dopamine in reinforcement learning. Much data argue, however, for a critical role for prefrontal cortex (PFC) dopamine in stimulus selection in attentional learning. Here, we present a new computational model that simulates performance in multicue category learning, such as the “weather prediction” task. The model addresses how PD and dopamine medications affect stimulus selection processes, which mediate reinforcement learning. In this model, PFC dopamine is key for attentional learning, whereas basal ganglia dopamine, consistent with other models, is key for reinforcement and motor learning. The model assumes that competitive dynamics among PFC neurons is the neural mechanism underlying stimulus selection with limited attentional resources, whereas competitive dynamics among striatal neurons is the neural mechanism underlying action selection. According to our model, PD is associated with decreased phasic and tonic dopamine levels in both PFC and basal ganglia. We assume that dopamine medications increase dopamine levels in both the basal ganglia and PFC, which, in turn, increase tonic dopamine levels but decrease the magnitude of phasic dopamine signaling in these brain structures. Increase of tonic dopamine levels in the simulated PFC enhances attentional shifting performance. The model provides a mechanistic account for several phenomena, including (a) medicated PD patients are more impaired at multicue probabilistic category learning than unmedicated patients and (b) medicated PD patients opt out of reversal when there are alternative and redundant cue dimensions.

[1]  Jonathan D. Cohen,et al.  On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory , 2007 .

[2]  J. Cohen,et al.  Context, cortex, and dopamine: a connectionist approach to behavior and biology in schizophrenia. , 1992, Psychological review.

[3]  M. Gluck,et al.  Dopaminergic Drugs Modulate Learning Rates and Perseveration in Parkinson's Patients in a Dynamic Foraging Task , 2009, The Journal of Neuroscience.

[4]  R. Prediger,et al.  The risk is in the air: Intranasal administration of MPTP to rats reproducing clinical features of Parkinson's disease , 2006, Experimental Neurology.

[5]  K. Jellinger Post mortem studies in Parkinson's disease--is it possible to detect brain areas for specific symptoms? , 1999, Journal of neural transmission. Supplementum.

[6]  M. Gluck,et al.  l-dopa impairs learning, but spares generalization, in Parkinson's disease , 2006, Neuropsychologia.

[7]  H Pashler,et al.  How persuasive is a good fit? A comment on theory testing. , 2000, Psychological review.

[8]  Mark D. Humphries,et al.  A robot model of the basal ganglia: Behavior and intrinsic processing , 2006, Neural Networks.

[9]  A. Amos A Computational Model of Information Processing in the Frontal Cortex and Basal Ganglia , 2000, Journal of Cognitive Neuroscience.

[10]  M. Gluck,et al.  A neurocomputational model of tonic and phasic dopamine in action selection: A comparison with cognitive deficits in Parkinson's disease , 2009, Behavioural Brain Research.

[11]  Mark A. Gluck,et al.  Dissociating the Cognitive Effects of Levodopa versus Dopamine Agonists in a Neurocomputational Model of Learning in Parkinson’s Disease , 2012, Neurodegenerative Diseases.

[12]  J. Kruschke Attention in Learning , 2003 .

[13]  J. Cohen,et al.  Context-processing deficits in schizophrenia: converging evidence from three theoretically motivated cognitive tasks. , 1999, Journal of abnormal psychology.

[14]  Russell A Poldrack,et al.  Cognition: Basal Ganglia Role , 2009 .

[15]  B. Dubois,et al.  Motivation, reward, and Parkinson’s disease: influence of dopatherapy , 2002, Neuropsychologia.

[16]  Daphna Shohamy,et al.  Distinct Hippocampal and Basal Ganglia Contributions to Probabilistic Learning and Reversal , 2009, Journal of Cognitive Neuroscience.

[17]  Fumitoshi Kodaka,et al.  Differential Contributions of Prefrontal and Hippocampal Dopamine D1 and D2 Receptors in Human Cognitive Functions , 2008, The Journal of Neuroscience.

[18]  W. T. Maddox,et al.  Characterizing rule-based category learning deficits in patients with Parkinson's disease , 2007, Neuropsychologia.

[19]  R. Sutherland,et al.  The hippocampal formation is necessary for rats to learn and remember configural discriminations , 1989, Behavioural Brain Research.

[20]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[21]  P. Stratta,et al.  Schizophrenic deficits in the processing of context. , 1998, Archives of general psychiatry.

[22]  Thomas E. Hazy,et al.  Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[23]  D. Lagnado,et al.  Medication impairs probabilistic classification learning in Parkinson's disease , 2010, Neuropsychologia.

[24]  P. Brown,et al.  Cortico-cortical coupling in Parkinson's disease and its modulation by therapy. , 2005, Brain : a journal of neurology.

[25]  Joseph P. Huston,et al.  l-DOPA and psychosis: Evidence for l-DOPA-induced increases in prefrontal cortex dopamine and in serum corticosterone , 1995, Biological Psychiatry.

[26]  John G. Taylor,et al.  A neural model of working memory processes in normal subjects, Parkinson's disease and schizophrenia for fMRI design and predictions , 2000, Neural Networks.

[27]  Gabriele Wulf,et al.  Motor learning benefits of self-controlled practice in persons with Parkinson's disease. , 2012, Gait & posture.

[28]  Jonathan D. Cohen,et al.  Computational perspectives on dopamine function in prefrontal cortex , 2002, Current Opinion in Neurobiology.

[29]  廣瀬雄一,et al.  Neuroscience , 2019, Workplace Attachments.

[30]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[31]  M. Gluck,et al.  Role of the basal ganglia in category learning: how do patients with Parkinson's disease learn? , 2004, Behavioral neuroscience.

[32]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[33]  John P. Aggleton,et al.  Intact negative patterning in rats with fornix or combined perirhinal and postrhinal cortex lesions , 2000, Experimental Brain Research.

[34]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[35]  J. Cohen,et al.  Schizophrenic deficits in the processing of context. A test of a theoretical model. , 1996, Archives of general psychiatry.

[36]  C. Da Cunha,et al.  Emotional, cognitive and neurochemical alterations in a premotor stage model of Parkinson's disease , 2008, Neuroscience.

[37]  M. Gluck,et al.  Basal ganglia and dopamine contributions to probabilistic category learning , 2008, Neuroscience & Biobehavioral Reviews.

[38]  F. Gonon,et al.  Cortical Inputs and GABA Interneurons Imbalance Projection Neurons in the Striatum of Parkinsonian Rats , 2006, The Journal of Neuroscience.

[39]  T. Robbins,et al.  Enhanced or impaired cognitive function in Parkinson's disease as a function of dopaminergic medication and task demands. , 2001, Cerebral cortex.

[40]  F. Gregory Ashby,et al.  FROST: A Distributed Neurocomputational Model of Working Memory Maintenance , 2005, Journal of Cognitive Neuroscience.

[41]  Glyn W. Humphreys,et al.  Connectionist models of neuropsychological disorders , 1997, Trends in Cognitive Sciences.

[42]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[43]  Sébastien Hélie,et al.  Simulating the effects of dopamine imbalance on cognition: From positive affect to Parkinson's disease , 2012, Neural Networks.

[44]  Anthony S. Maida,et al.  Using TD learning to simulate working memory performance in a model of the prefrontal cortex and basal ganglia , 2007, Cognitive Systems Research.

[45]  Michael J. Frank,et al.  A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism , 2008, Neuropsychologia.

[46]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[47]  Sébastien Hélie,et al.  A neurocomputational account of cognitive deficits in Parkinson's disease , 2012, Neuropsychologia.

[48]  R. Shepard,et al.  Learning and memorization of classifications. , 1961 .

[49]  John G. Taylor,et al.  A Hard Wired Model of Coupled Frontal Working memories for various Tasks , 1999, Inf. Sci..

[50]  C. Myers,et al.  A model of amygdala–hippocampal–prefrontal interaction in fear conditioning and extinction in animals , 2013, Brain and Cognition.

[51]  T. Robbins,et al.  Contrasting mechanisms of impaired attentional set-shifting in patients with frontal lobe damage or Parkinson's disease. , 1993, Brain : a journal of neurology.

[52]  M. Gluck,et al.  Neural Mechanisms Underlying Probabilistic Category Learning in Normal Aging , 2005, The Journal of Neuroscience.

[53]  Mony J de Leon,et al.  Hippocampal Atrophy Disrupts Transfer Generalization in Nondemented Elderly , 2002, Journal of geriatric psychiatry and neurology.

[54]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[55]  T. Robbins,et al.  Dopaminergic basis for deficits in working memory but not attentional set-shifting in Parkinson's disease , 2005, Neuropsychologia.

[56]  S. Kish,et al.  Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease. Pathophysiologic and clinical implications. , 1988, The New England journal of medicine.

[57]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[58]  Juha O. Rinne,et al.  Personality traits and brain dopaminergic function in Parkinson's disease , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[59]  John K. Kruschke,et al.  Learning Involves Attention , 2000 .

[60]  T. Stafford,et al.  Biologically constrained action selection improves cognitive control in a model of the Stroop task , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[61]  Mark S. Seidenberg,et al.  Impairments in verb morphology after brain injury: a connectionist model. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[62]  Cameron S. Carter,et al.  Dopamine and the mechanisms of cognition: Part I. A neural network model predicting dopamine effects on selective attention , 1998, Biological Psychiatry.

[63]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[64]  Erin Kendall Braun,et al.  A Trade-Off between Feedback-Based Learning and Episodic Memory for Feedback Events: Evidence from Parkinson’s Disease , 2012, Neurodegenerative Diseases.

[65]  W. Schultz Multiple dopamine functions at different time courses. , 2007, Annual review of neuroscience.

[66]  Michiyo Iba,et al.  Involvement of the dorsolateral prefrontal cortex of monkeys in visuospatial target selection. , 2003, Journal of neurophysiology.

[67]  Joel L. Davis,et al.  In : Models of Information Processing in the Basal Ganglia , 2008 .

[68]  M. Gluck,et al.  How do people solve the "weather prediction" task?: individual variability in strategies for probabilistic category learning. , 2002, Learning & memory.

[69]  James C. Houk,et al.  Information Processing in Modular Circuits Linking Basal Ganglia and Cerebral Cortex , 1994 .

[70]  M. Gluck,et al.  Hippocampal mediation of stimulus representation: A computational theory , 1993, Hippocampus.