Shruti-agent: a structured connectionist architecture for reasoning and decision-making

One of the great scientific challenges of today is to understand how complex cognitive processes such as inference and decision-making can be achieved by the human brain. The incredible complexity of the brain makes computational modeling an indispensable tool to meet this challenge. Connectionist models that explain cognition in terms of brain-like structures and mechanisms are particularly useful in this endeavor. This thesis describes SHRUTI-agent, a neurally motivated, structured connectionist agent architecture that is capable of inference, decision-making, and interaction with a simulated environment. SHRUTI-agent is an extension of SHRUTI, a connectionist model that demonstrates how temporal synchrony variable binding in conjunction with structured, neurally plausible representations can support several aspects of high-level cognition. Creating a decision-making agent architecture based on SHRUTI involved a number of enhancements and additions to the SHRUTI model, and these comprise the primary contributions of the thesis. As a first step towards building an effective system for inference and decision-making, the evidential reasoning capability of the SHRUTI model was significantly enhanced. The resulting model demonstrates that relational knowledge and approximately probabilistic reasoning can be effectively combined in a connectionist architecture. Simple or reactive decision-making was made possible by the introduction into the model of compatible connectionist structures and mechanisms to represent and manipulate utility. It is shown that the basic decision-making capability of the model has much in common with formal methods from artificial intelligence. In order to enable the system to handle complex or sequential decision problems, a set of control mechanisms was developed. It is shown that the complex control mechanisms required for decision-making (e.g. hypothesis-testing) can be based on combinations of simple control primitives such as monitoring, filtering, and maintenance. Each of these control primitives is described in terms of its connectionist realization. Moreover, a subset of these control primitives are used as the basis for an implemented model of working memory in lateral prefrontal cortex. Operation of the SHRUTI-agent model is illustrated in two case studies, one involving simulated robot soccer and the other involving a military domain. Finally, consideration of the impact of learning on decision-making performance led to some preliminary work aimed at understanding how SHRUTI's structured representations can be learned. Utile concept learning is introduced as a mechanism for recruitment of new concepts and rules that lead to improved decision-making performance, and causal Hebbian learning is introduced as a mechanism for learning statistical causal rules.

[1]  M. W. Brown,et al.  Episodic memory, amnesia, and the hippocampal–anterior thalamic axis , 1999, Behavioral and Brain Sciences.

[2]  J. Jonides,et al.  Storage and executive processes in the frontal lobes. , 1999, Science.

[3]  Michael G. Dyer,et al.  High-level Inferencing in a Connectionist Network , 1989 .

[4]  Allen Newell,et al.  Elements of a theory of human problem solving. , 1958 .

[5]  J. Grafman,et al.  Are the frontal lobes implicated in “planning” functions? Interpreting data from the Tower of Hanoi , 1995, Neuropsychologia.

[6]  Clayton E Curtis,et al.  Organization of working memory within the human prefrontal cortex: a PET study of self-ordered object working memory , 2000, Neuropsychologia.

[7]  S. Grossberg,et al.  Neural dynamics of decision making under risk: affective balance and cognitive-emotional interactions. , 1988, Psychological review.

[8]  B. Postle,et al.  Maintenance versus Manipulation of Information Held in Working Memory: An Event-Related fMRI Study , 1999, Brain and Cognition.

[9]  L. Shastri,et al.  From simple associations to systematic reasoning: A connectionist representation of rules, variables and dynamic bindings using temporal synchrony , 1993, Behavioral and Brain Sciences.

[10]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[11]  R. Weller Two cortical visual systems in Old World and New World primates. , 1988, Progress in brain research.

[12]  C. Kase,et al.  Global spatial disorientation Clinico-pathologic correlations , 1977, Journal of the Neurological Sciences.

[13]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[14]  D. Stuss,et al.  The Frontal Lobes , 1986 .

[15]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[16]  H. Raiffa,et al.  Decisions with Multiple Objectives , 1993 .

[17]  James L Olds,et al.  Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. , 1954, Journal of comparative and physiological psychology.

[18]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[19]  J. Duncan,et al.  Common regions of the human frontal lobe recruited by diverse cognitive demands , 2000, Trends in Neurosciences.

[20]  P. Schoemaker,et al.  Prospect theory's reflection hypothesis: A critical examination , 1980 .

[21]  John E. Hummel,et al.  Distributed representations of structure: A theory of analogical access and mapping. , 1997 .

[22]  A. Paans,et al.  Brain Activation Related to the Representations of External Space and Body Scheme in Visuomotor Control , 2001, NeuroImage.

[23]  Risto Miikkulainen,et al.  Natural Language Processingwith Modular Neural Networks and Distributed Lexicon , 1991 .

[24]  Nevin L. Zhang,et al.  A simple approach to Bayesian network computations , 1994 .

[25]  Marcelo Simoes Introduction to Fuzzy Control , 2003 .

[26]  D. Buonomano,et al.  Cortical plasticity: from synapses to maps. , 1998, Annual review of neuroscience.

[27]  Michael C. Anderson,et al.  Inhibitory processes and the control of memory retrieval , 2002, Trends in Cognitive Sciences.

[28]  D. R. Mani,et al.  Reflexive Reasoning with Multiple Instantiation in a Connectionist Reasoning System with a Type Hierarchy , 1993 .

[29]  D. Kieras,et al.  Modern computational perspectives on executive mental processes and cognitive control: Where to from here? , 2000 .

[30]  R. Nicoll,et al.  Long-term potentiation--a decade of progress? , 1999, Science.

[31]  John R. Anderson,et al.  A Production System Theory of Serial Memory , 1997 .

[32]  Daniel T. Levin,et al.  Animals and Artifacts May Not Be Treated Equally: Differentiating Strong and Weak Forms of Category-Specific Visual Agnosia , 2001, Brain and Cognition.

[33]  G. Bower,et al.  Rewarding and punishing effects from stimulating the same place in the rat's brain. , 1958, Journal of comparative and physiological psychology.

[34]  Allen Newell,et al.  Computer science as empirical inquiry: symbols and search , 1976, CACM.

[35]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[36]  W. Singer,et al.  Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  R. Shadmehr,et al.  Neural correlates of motor memory consolidation. , 1997, Science.

[38]  James L. McClelland,et al.  On the control of automatic processes: a parallel distributed processing account of the Stroop effect. , 1990, Psychological review.

[39]  E. Rolls,et al.  Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. , 1996, Journal of neurophysiology.

[40]  Lynn C. Robertson,et al.  Implicit Representations of Space after Bilateral Parietal Lobe Damage , 2001, Journal of Cognitive Neuroscience.

[41]  B. Milner,et al.  Deficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man , 1982, Neuropsychologia.

[42]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[43]  T. Bliss Long-lasting potentiation of synaptic transmission , 2005 .

[44]  Gregory P. Lee,et al.  Different Contributions of the Human Amygdala and Ventromedial Prefrontal Cortex to Decision-Making , 1999, The Journal of Neuroscience.

[45]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[46]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[47]  R. Knight,et al.  Prefrontal cortex regulates inhibition and excitation in distributed neural networks. , 1999, Acta psychologica.

[48]  Antony Browne,et al.  Connectionist inference models , 2001, Neural Networks.

[49]  S. Grossberg,et al.  Pattern formation, contrast control, and oscillations in the short term memory of shunting on-center off-surround networks , 1975, Biological Cybernetics.

[50]  Craig Boutilier,et al.  Computing Optimal Policies for Partially Observable Decision Processes Using Compact Representations , 1996, AAAI/IAAI, Vol. 2.

[51]  M. Petrides Comparative architectonic analysis of the human and the macaque frontal cortex , 1994 .

[52]  Craig Boutilier,et al.  Exploiting Structure in Policy Construction , 1995, IJCAI.

[53]  John Jonides,et al.  Order Information in Working Memory: fMRI Evidence for Parietal and Prefrontal Mechanisms , 2000, Journal of Cognitive Neuroscience.

[54]  Jonathan Baron,et al.  Behavioral Law and Economics: Reluctance to Vaccinate: Omission Bias and Ambiguity , 1990 .

[55]  Paul Thagard,et al.  Analogical Mapping by Constraint Satisfaction , 1989, Cogn. Sci..

[56]  Lokendra Shastri,et al.  Learning structured representations , 2003, Neurocomputing.

[57]  Patrick D. Roberts,et al.  Spike timing dependent synaptic plasticity in biological systems , 2002, Biological Cybernetics.

[58]  Alan C. Evans,et al.  Changes in brain activity related to eating chocolate: from pleasure to aversion. , 2001, Brain : a journal of neurology.

[59]  E T Rolls,et al.  Representations of pleasant and painful touch in the human orbitofrontal and cingulate cortices. , 2003, Cerebral cortex.

[60]  R. Morris,et al.  Neural correlates of planning ability: Frontal lobe activation during the tower of London test , 1993, Neuropsychologia.

[61]  Seth Bullock,et al.  Simple Heuristics That Make Us Smart , 1999 .

[62]  A. Tversky,et al.  Contingent weighting in judgment and choice , 1988 .

[63]  Leslie G. Ungerleider,et al.  ‘What’ and ‘where’ in the human brain , 1994, Current Opinion in Neurobiology.

[64]  E. Rolls,et al.  Representation of pleasant and aversive taste in the human brain. , 2001, Journal of neurophysiology.

[65]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[66]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[67]  Yun Peng,et al.  A Neural Network Approach to Approximating Map in Belief Networks , 2002, Int. J. Neural Syst..

[68]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[69]  Craig Boutilier,et al.  Decision-Theoretic Planning: Structural Assumptions and Computational Leverage , 1999, J. Artif. Intell. Res..

[70]  Nicholas Kushmerick,et al.  An Algorithm for Probabilistic Planning , 1995, Artif. Intell..

[71]  John R. Anderson,et al.  The Transfer of Cognitive Skill , 1989 .

[72]  Craig Boutilier,et al.  Symbolic Dynamic Programming for First-Order MDPs , 2001, IJCAI.

[73]  P. Thagard,et al.  Inference to the best plan: A coherence theory of decision. , 1997 .

[74]  Lokendra Shastri,et al.  Soft Computing in SHRUTI - A Neurally Plausible Model of Reflexive Reasoning and Relational Information Processing , 1999, IIA/SOCO.

[75]  Daphne Koller,et al.  Computing Factored Value Functions for Policies in Structured MDPs , 1999, IJCAI.

[76]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[77]  Edward H. Shortliffe,et al.  Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project (The Addison-Wesley series in artificial intelligence) , 1984 .

[78]  Robert P. Goldman,et al.  From knowledge bases to decision models , 1992, The Knowledge Engineering Review.

[79]  N. Costes,et al.  Brain processing of visual sexual stimuli in human males , 2000, Human brain mapping.

[80]  N Butters,et al.  Retention of Delayed-Alternation: Effect of Selective Lesions of Sulcus Principalis , 1969, Science.

[81]  J. Haxby,et al.  Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects , 1999, Nature Neuroscience.

[82]  H. Breiter,et al.  Reward Circuitry Activation by Noxious Thermal Stimuli , 2001, Neuron.

[83]  Gary F. Marcus,et al.  Plasticity and Nativism: Towards a Resolution of an Apparent Paradox , 2001, Emergent Neural Computational Architectures Based on Neuroscience.

[84]  M. Just,et al.  From the SelectedWorks of Marcel Adam Just 1992 A capacity theory of comprehension : Individual differences in working memory , 2017 .

[85]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[86]  P. Goldman-Rakic,et al.  Dissociation of object and spatial processing domains in primate prefrontal cortex. , 1993, Science.

[87]  Mortimer Mishkin,et al.  A re-examination of the effects of frontal lesions on object alternation , 1969 .

[88]  Nigel Goddard,et al.  Rochester Connectionist Simulator , 1989 .

[89]  Lokendra Shastri,et al.  A Connectionist Encoding of Parameterized Schemas and Reactive Plans , 2002 .

[90]  Jonathan D. Cohen,et al.  A Biologically Based Computational Model of Working Memory , 1999 .

[91]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[92]  A. Baddeley,et al.  Dual-task performance in dysexecutive and nondysexecutive patients with a frontal lesion. , 1997, Neuropsychology.

[93]  E S BOYD,et al.  Positive and Negative Reinforcement from Intracranial Stimulation of a Teleost , 1962, Science.

[94]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[95]  Kenneth D. Miller,et al.  The Role of Constraints in Hebbian Learning , 1994, Neural Computation.

[96]  Alan C. Evans,et al.  Evidence for a two-stage model of spatial working memory processing within the lateral frontal cortex: a positron emission tomography study. , 1996, Cerebral cortex.

[97]  Richard E Thompson,et al.  Cerebellar circuits and synaptic mechanisms involved in classical eyeblink conditioning , 1997, Trends in Neurosciences.

[98]  P. Milner,et al.  Preconceptions and prerequisites: Understanding the function of synaptic plasticity will also depend on a better systems-level understanding of the multiple types of memory , 1997, Behavioral and Brain Sciences.

[99]  L. Shastri,et al.  A Connectionist Treatment of Negation and Inconsistency , 1996 .

[100]  Jonathan Baron,et al.  Decision-making biases in children and early adolescents: exploratory studies , 1993 .

[101]  L. Shastri,et al.  Acquisition of concepts and causal rules in SHRUTI , 2003 .

[102]  Robert M. French,et al.  A Connectionist Account of Interference Effects in Early Infant Memory and Categorization , 1997 .

[103]  Lokendra Shastri,et al.  Probabilistic Inference and Learning in a Connectionist Causal Network , 2000 .

[104]  D. Norman,et al.  Attention to action: Willed and automatic control , 1980 .

[105]  J. Lilly,et al.  Operant conditioning of the bottlenose dolphin with electrical stimulation of the brain. , 1962 .

[106]  Michael I. Jordan,et al.  Graphical models: Probabilistic inference , 2002 .

[107]  Fahiem Bacchus,et al.  Graphical models for preference and utility , 1995, UAI.

[108]  Jodie A. Baird,et al.  Executive Attention and Metacognitive Regulation , 2000, Consciousness and Cognition.

[109]  Ron Sun,et al.  A Hybrid Architecture for Situated Learning of Reactive Sequential Decision Making , 1999, Applied Intelligence.

[110]  M. Botvinick,et al.  The Contribution of the Anterior Cingulate Cortex to Executive Processes in Cognition , 1999, Reviews in the neurosciences.

[111]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[112]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[113]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[114]  W. Singer,et al.  Interhemispheric synchronization of oscillatory neuronal responses in cat visual cortex , 1991, Science.

[115]  D. Scott,et al.  Assigning Probabilities to Logical Formulas , 1966 .