The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers

Cognitive control has long been one of the most active areas of computational modeling work in cognitive science. The focus on computational models as a medium for specifying and developing theory predates the PDP books, and cognitive control was not one of the areas on which they focused. However, the framework they provided has injected work on cognitive control with new energy and new ideas. On the occasion of the books' anniversary, we review computational modeling in the study of cognitive control, with a focus on the influence that the PDP approach has brought to bear in this area. Rather than providing a comprehensive review, we offer a framework for thinking about past and future modeling efforts in this domain. We define control in terms of the optimal parameterization of task processing. From this vantage point, the development of control systems in the brain can be seen as responding to the structure of naturalistic tasks, through the filter of the brain systems with which control directly interfaces. This perspective lays open a set of fascinating but difficult research questions, which together define an important frontier for future computational research.

[1]  K. Lashley.  The problem of serial order in behavior , 1951 .

[2]  J. Knott.  The organization of behavior: A neuropsychological theory , 1951 .

[3]  A. Welford.  THE ‘PSYCHOLOGICAL REFRACTORY PERIOD’ AND THE TIMING OF HIGH‐SPEED PERFORMANCE—A REVIEW AND A THEORY , 1952 .

[4]  I. Kaufman,et al.  Cerebral Mechanisms in Behavior. The Hixon Symposium , 1953 .

[5]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[6]  D. Berlyne.  Uncertainty and conflict: a point of contact between information-theory and behavior-theory concepts. , 1957, Psychology Review.

[7]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[8]  G. Miller,et al.  Plans and the structure of behavior , 1960 .

[9]  B. Milner.  Effects of Different Brain Lesions on Card Sorting: The Role of the Frontal Lobes , 1963 .

[10]  P. Rabbitt.  Errors and error correction in choice-response tasks. , 1966, Journal of experimental psychology.

[11]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[12]  A. Battersby.  Plans and the Structure of Behavior , 1968 .

[13]  G. E. Alexander,et al.  Neuron Activity Related to Short-Term Memory , 1971, Science.

[14]  C. Blakemore.  The Working Brain , 1972, Nature.

[15]  M. Posner,et al.  Attention and cognitive control. , 1975 .

[16]  Alan S. Brown,et al.  Information Processing and Cognition: The Loyola Symposium , 1976 .

[17]  G. Schwartz,et al.  Consciousness and Self-Regulation , 1976 .

[18]  Daniel Gopher,et al.  On the Economy of the Human Processing System: A Model of Multiple Capacity. , 1977 .

[19]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[20]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[21]  D. Laming.  Choice reaction performance following an error , 1979 .

[22]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[23]  G. Miller,et al.  Cognitive science. , 1981, Science.

[24]  T. Shallice.  Specific impairments of planning. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[25]  B. Milner,et al.  Deficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man , 1982, Neuropsychologia.

[26]  F. Lhermitte.  'Utilization behaviour' and its relation to lesions of the frontal lobes. , 1983, Brain : a journal of neurology.

[27]  G. Logan,et al.  Costs and benefits of strategy construction in a speeded discrimination task , 1983, Memory & cognition.

[28]  H Pashler,et al.  Processing stages in overlapping tasks: evidence for a central bottleneck. , 1984, Journal of experimental psychology. Human perception and performance.

[29]  G. Logan.  Skill and automaticity: Relations, implications, and future directions. , 1985 .

[30]  J. Fuster.  The prefrontal cortex, mediator of cross-temporal contingencies. , 1985, Human neurobiology.

[31]  J. Duncan.  Disorganisation of behaviour after frontal lobe damage , 1986 .

[32]  Donald A. Norman,et al.  Attention to Action , 1986 .

[33]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[34]  O. G. Selfridge,et al.  Pandemonium: a paradigm for learning , 1988 .

[35]  Colin M. Macleod,et al.  Training and Stroop-like interference: evidence for a continuum of automaticity. , 1988, Journal of experimental psychology. Learning, memory, and cognition.

[36]  G. F. Tremblay,et al.  The Prefrontal Cortex , 1989, Neurology.

[37]  J. Changeux,et al.  A Simple Model of Prefrontal Cortex Function in Delayed-Response Tasks , 1989, Journal of Cognitive Neuroscience.

[38]  John R. Anderson.  The Adaptive Character of Thought , 1990 .

[39]  J D Cohen,et al.  A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. , 1990, Science.

[40]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[41]  T. Robbins,et al.  Extra-dimensional versus intra-dimensional set shifting performance following frontal lobe excisions, temporal lobe excisions or amygdalo-hippocampectomy in man , 1991, Neuropsychologia.

[42]  Colin M. Macleod.  Half a century of research on the Stroop effect: an integrative review. , 1991, Psychological bulletin.

[43]  J. Changeux,et al.  The Wisconsin Card Sorting Test: theoretical analysis and modeling in a neuronal network. , 1991, Cerebral cortex.

[44]  A. Benton,et al.  Frontal Lobe Function and Dysfunction , 1991 .

[45]  J. Cohen,et al.  Context, cortex, and dopamine: a connectionist approach to behavior and biology in schizophrenia. , 1992, Psychological review.

[46]  E. Donchin,et al.  Optimizing the use of information: strategic control of activation of responses. , 1992, Journal of experimental psychology. General.

[47]  A. Henik,et al.  Controlling Stroop effects by manipulating expectations for color words , 1992, Memory & cognition.

[48]  John R. Anderson,et al.  The Adaptive Character of Thought , 1990 .

[49]  J. Fuster.  Frontal lobes , 1993, Current Opinion in Neurobiology.

[50]  R. Passingham.  The frontal lobes and voluntary action , 1993 .

[51]  D. Meyer,et al.  A Neural System for Error Detection and Compensation , 1993 .

[52]  D. Zipser,et al.  A spiking network model of short-term active memory , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[53]  P. Gollwitzer.  Goal Achievement: The Role of Intentions , 1993 .

[54]  Jonathan D. Cohen,et al.  A Computational Model of Prefrontal Cortex Function , 1994, NIPS.

[55]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[56]  J. Hohnsbein,et al.  Event-related potential correlates of errors in reaction tasks. , 1995, Electroencephalography and clinical neurophysiology. Supplement.

[57]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[58]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[59]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[60]  Jonathan D. Cohen,et al.  A preliminary theory of the interactions between prefrontal cortex and hippocampus that contribute to planning and prospective memory. , 1996 .

[61]  R. O’Reilly,et al.  A computational approach to prefrontal cortex, cognitive control and schizophrenia: recent developments and current challenges. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[62]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[63]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[64]  S Dehaene,et al.  A hierarchical neuronal network for planning behavior. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[65]  D E Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. , 1997, Psychological review.

[66]  Edward E. Smith,et al.  Temporal dynamics of brain activation during a working memory task , 1997, Nature.

[67]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[68]  David E. Kieras,et al.  A computational theory of executive cognitive processes and multiple-task performance: Part 2. Accounts of psychological refractory-period phenomena. , 1997 .

[69]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[70]  R. Knight.  Distributed Cortical Network for Visual Attention , 1997, Journal of Cognitive Neuroscience.

[71]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[72]  T. Paus,et al.  Regional differences in the effects of task difficulty and motor output on blood flow response in the human anterior cingulate cortex: a review of 107 PET activation studies , 1998, Neuroreport.

[73]  Leslie Pack Kaelbling,et al.  Learning Policies with External Memory , 1999, ICML.

[74]  Jonathan D. Cohen,et al.  Conflict monitoring versus selection-for-action in anterior cingulate cortex , 1999, Nature.

[75]  Jonathan D. Cohen,et al.  Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function , 1999, Biological Psychiatry.

[76]  M. Botvinick,et al.  Parsing executive processes: strategic vs. evaluative functions of the anterior cingulate cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[77]  N. Cohen,et al.  Prefrontal regions play a predominant role in imposing an attentional 'set': evidence from fMRI. , 2000, Brain research. Cognitive brain research.

[78]  T. Shallice,et al.  CONTENTION SCHEDULING AND THE CONTROL OF ROUTINE ACTIVITIES , 2000, Cognitive neuropsychology.

[79]  J. Duncan,et al.  Common regions of the human frontal lobe recruited by diverse cognitive demands , 2000, Trends in Neurosciences.

[80]  J. Cohen,et al.  Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. , 2000, Science.

[81]  James L. McClelland,et al.  Stipulating versus discovering representations , 2000, Behavioral and Brain Sciences.

[82]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[83]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[84]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[85]  J. C. Johnston,et al.  Attention and performance. , 2001, Annual review of psychology.

[86]  J. Ridley.  Studies of Interference in Serial Verbal Reactions , 2001 .

[87]  Jonathan D. Cohen,et al.  Computational perspectives on dopamine function in prefrontal cortex , 2002, Current Opinion in Neurobiology.

[88]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[89]  Jonathan D. Cohen,et al.  Prefrontal cortex and dynamic categorization tasks: representational organization and neuromodulatory control. , 2002, Cerebral cortex.

[90]  Marc W. Howard,et al.  A distributed representation of temporal context , 2002 .

[91]  D. Plaut,et al.  Representing task context: proposals based on a connectionist model of action , 2002, Psychological research.

[92]  N. Yeung,et al.  Switching between tasks of unequal familiarity: the role of stimulus-attribute and response-set selection. , 2003, Journal of experimental psychology. Human perception and performance.

[93]  J. Seamans,et al.  The principal features and mechanisms of dopamine modulation in the prefrontal cortex , 2004, Progress in Neurobiology.

[94]  Jonathan D. Cohen,et al.  Conflict monitoring and anterior cingulate cortex: an update , 2004, Trends in Cognitive Sciences.

[95]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[96]  Mark S. Gilzenrat,et al.  A Systems-Level Perspective on Attention and Cognitive Control: Guided Activation, Adaptive Gating, Conflict Monitoring, and Exploitation versus Exploration. , 2004 .

[97]  Emilio Salinas,et al.  Fast Remapping of Sensory Stimuli onto Motor Actions on the Basis of Contextual Modulation , 2004, The Journal of Neuroscience.

[98]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[99]  Jonathan D. Cohen,et al.  The neural basis of error detection: conflict monitoring and the error-related negativity. , 2004, Psychological review.

[100]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[101]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[102]  Clay B. Holroyd,et al.  A mechanism for error detection in speeded response time tasks. , 2005, Journal of experimental psychology. General.

[103]  Alicia P. Wolfe,et al.  Identifying useful subgoals in reinforcement learning by local graph partitioning , 2005, ICML.

[104]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[105]  Lauren M. Bylsma,et al.  The conflict adaptation effect: It’s not just priming , 2005, Cognitive, affective & behavioral neuroscience.

[106]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[107]  Jun Tani,et al.  How Hierarchical Control Self-organizes in Artificial Adaptive Systems , 2005, Adapt. Behav..

[108]  J. Tenenbaum,et al.  Opinion TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Theory-based Bayesian models of inductive learning and reasoning , 2022 .

[109]  Philip Holmes,et al.  Rapid decision threshold modulation by reward rate in a neural network , 2006, Neural Networks.

[110]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[111]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[112]  M. Botvinick.  Conflict monitoring and decision making: Reconciling two perspectives on anterior cingulate function , 2007, Cognitive, affective & behavioral neuroscience.

[113]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[114]  T. Braver,et al.  Explaining the many varieties of working memory variation: Dual mechanisms of cognitive control. , 2007 .

[115]  T. Aflalo,et al.  Mapping Behavioral Repertoire onto the Cortex , 2007, Neuron.

[116]  Matthew M Botvinick,et al.  Multilevel structure in behaviour and in the brain: a model of Fuster's hierarchy , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[117]  Jonathan D. Cohen,et al.  On the Control of Control: The Role of Dopamine in Regulating Prefrontal Function and Working Memory , 2007 .

[118]  Peter Dayan,et al.  Bilinearity, Rules, and Prefrontal Cortex , 2007, Frontiers Comput. Neurosci..

[119]  John R. Anderson,et al.  Using fMRI to Test Models of Complex Cognition , 2008, Cogn. Sci..

[120]  Andrew G. Barto,et al.  Behavioral building blocks for autonomous agents: description, identification, and learning , 2008 .

[121]  Jonathan D. Cohen,et al.  Learning to Use Working Memory in Partially Observable Environments through Dopaminergic Reinforcement , 2008, NIPS.

[122]  M. Botvinick.  Hierarchical models of behavior and prefrontal function , 2008, Trends in Cognitive Sciences.

[123]  Sean M. Polyn,et al.  Memory search and the neural representation of context , 2008, Trends in Cognitive Sciences.

[124]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[125]  M. D’Esposito.  Working memory. , 2008, Handbook of clinical neurology.

[126]  Jeremy R. Reynolds,et al.  Developing PFC representations using reinforcement learning , 2009, Cognition.

[127]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[128]  James L. McClelland.  Running Head : Letting Structure Emerge Letting Structure Emerge : Connectionist and Dynamical Systems Approaches to Understanding Cognition , 2009 .

[129]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[130]  G. Pezzulo,et al.  Thinking as the control of imagination: a conceptual framework for goal-directed systems , 2009, Psychological research.

[131]  Jonathan D. Cohen,et al.  Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. , 2009, Journal of experimental psychology. Human perception and performance.

[132]  Matthew M Botvinick,et al.  Empirical and computational support for context-dependent representations of serial order: reply to Bowers, Damian, and Davis (2009). , 2009, Psychological review.

[133]  Lauren M. Bylsma,et al.  Toward an integrated account of object and action selection: A computational analysis and empirical findings from reaching-to-grasp and tool-use , 2009, Neuropsychologia.

[134]  Sean M. Polyn,et al.  A context maintenance and retrieval model of organizational processes in free recall. , 2009, Psychological review.

[135]  Wolfgang M. Pauli,et al.  Computational models of cognitive control , 2010, Current Opinion in Neurobiology.

[136]  J. Duncan.  The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour , 2010, Trends in Cognitive Sciences.

[137]  Jonathan D. Cohen,et al.  Do humans produce the speed–accuracy trade-off that maximizes reward rate? , 2010, Quarterly journal of experimental psychology.

[138]  P. Berkes,et al.  Statistically Optimal Perception and Learning: from Behavior to Neural Representations , 2022 .

[139]  Hannah S. Locke,et al.  Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts , 2010, Proceedings of the National Academy of Sciences.

[140]  C. Ranganath,et al.  Prefrontal and Medial Temporal Lobe Activity at Encoding Predicts Temporal Context Memory , 2010, The Journal of Neuroscience.

[141]  Joseph T. McGuire,et al.  Prefrontal cortex, cognitive control, and the registration of decision costs , 2010, Proceedings of the National Academy of Sciences.

[142]  Michael W. Cole,et al.  Conflict over Cingulate Cortex: Between-Species Differences in Cingulate May Support Enhanced Cognitive Flexibility in Humans , 2010, Brain, Behavior and Evolution.

[143]  Joseph T. McGuire,et al.  Decision making and the avoidance of cognitive demand. , 2010, Journal of experimental psychology. General.

[144]  James L. McClelland,et al.  Letting structure emerge: connectionist and dynamical systems approaches to cognition , 2010, Trends in Cognitive Sciences.

[145]  Richard P. Cooper,et al.  Cognitive Control: Componential or Emergent? , 2010, Top. Cogn. Sci..

[146]  Daniel A. Braun,et al.  Structure learning in action , 2010, Behavioural Brain Research.

[147]  Joshua W. Brown,et al.  Medial prefrontal cortex as an action-outcome predictor , 2011, Nature Neuroscience.

[148]  P. Goldman-Rakic.  Circuitry of Primate Prefrontal Cortex and Regulation of Behavior by Representational Memory , 2011 .

[149]  Andrew M. Saxe,et al.  Acquisition of decision making criteria: reward rate ultimately beats accuracy , 2011, Attention, perception & psychophysics.

[150]  Michael W. Cole,et al.  Rapid Transfer of Abstract Rules to Novel Contexts in Human Lateral Prefrontal Cortex , 2011, Front. Hum. Neurosci..

[151]  M. Botvinick.  Hierarchical reinforcement learning and decision making , 2012, Current Opinion in Neurobiology.

[152]  M. Botvinick,et al.  Neural representations of events arise from temporal community structure , 2013, Nature Neuroscience.

[153]  Jonathan D. Cohen,et al.  The Expected Value of Control: An Integrative Theory of Anterior Cingulate Cortex Function , 2013, Neuron.

[154]  Joseph T. McGuire,et al.  Neural and Behavioral Evidence for an Intrinsic Cost of Self-Control , 2013, PloS one.

[155]  Jonathan D. Cohen,et al.  Indirection and symbol-like processing in the prefrontal cortex and basal ganglia , 2013, Proceedings of the National Academy of Sciences.

[156]  Angela J. Yu,et al.  Bayesian Prediction and Evaluation in the Anterior Cingulate Cortex , 2013, The Journal of Neuroscience.

[157]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[158]  Alec Solway,et al.  Optimal Behavioral Hierarchy , 2014, PLoS Comput. Biol..

[159]  M. Botvinick,et al.  A labor/leisure tradeoff in cognitive control. , 2014, Journal of experimental psychology. General.

[160]  J D Cohen,et al.  Multitasking versus multiplexing: Toward a normative account of limitations in the simultaneous execution of control-demanding behaviors , 2014, Cognitive, affective & behavioral neuroscience.

[161]  D. Harrison.  Frontal Lobe Syndromes , 2015 .

[162]  Benjamin Naumann.  The Architecture Of Cognition , 2016 .

[163]  Yvonne Koch.  The Psychology Of Learning And Motivation , 2016 .

[164]  Jonathan D. Cohen,et al.  Prefrontal Cortex and the Flexibility of Cognitive Control : Rules Without Symbols , 2022 .