Dynamic decision making: neuronal, computational, and cognitive underpinnings

Dynamic decision making: neuronal, computational, and cognitive underpinnings Peter Dayan (dayan@gatsby.ucl.ac.uk) Gatsby computational neuroscience unit, University College London Nigel Harvey (n.harvey@ucl.ac.uk), Maarten Speekenbrink* (m.speekenbrink@ucl.ac.uk) Cognitive, Perceptual and Brain Sciences, University College London Magda Osman* (m.osman@qmul.ac.uk) Experimental Biology and Psychology Centre, Queen Mary University Masataka Watanabe (watanabe-ms@igakuken.or.jp) Department of Physiological Psychology, Tokyo Metropolitan Institute of Medical Science Keywords: Dynamic Decision Making, Computational models of reinforcement learning, Animal learning, Applied decision making Challenging Issues As complexity in our everyday environment increases (e.g., mobile applications for monitoring energy consumption), how do we adapt and react to the changing demands placed on us? In dynamic decision making (DDM) problems, the environment changes over time due to previous decisions made and/or factors outside the control of the decision-maker. To maximize his/her reward, an agent effectively needs to control a complex dynamic system. This often involves planning in the face of uncertainty about how decisions change the state of the system and the rewards that can be obtained. Thus, DDM refers to a process by which an agent selects a course of action in a manner that achieves or maintains a desired state in a dynamic environment. This includes balancing exploration and exploitation, distinguishing between different sources of variability within the environment, and tracking the current state of the environment (i.e., filtering). Thus far there has been little attempt at a synthesis of the amassing research from different areas of cognitive science directed towards understanding DDM. The objective of this symposium is to bring together the latest theoretical approaches and empirical research investigating DDM. The speakers range in expertise from comparative (Prof Watanabe), applied (Prof Harvey) and cognitive psychology (Dr Osman), computational neuroscience (Prof Dayan), and computational learning theory (Dr Speekenbrink). By bringing these diverse approaches together, the aim is to generate discussion around the following critical question: What are the processes/mechanisms that enable us to adapt to changes in uncertain environments in terms of the information we process, the decisions we make, and the intrinsic and extrinsic goals that we pursue? The symposium will consist of a general introduction (Osman), three talks (Dayan, Harvey, Wanatabe) and an extended discussion (moderated by Speekenbrink) involving all participants. Peter Dayan Peter Dayan’s work in computational and experimental neuroscience has contributed significantly to our understanding of the neural mechanisms underlying DDM and the learning of reward structures therein. Dayan is an expert on reinforcement learning and in recent work, has elucidated the distinction between “model-based” and “model-free” learning and the neural circuits supporting these. Model-based learning, usually associated with declarative task-knowledge, can support complex planning. Model-free learning, due to its more procedural nature, supports quick and habitual decisions, but will not cope well in an environment that undergoes rapid, abrupt changes. Dayan's recent work has shown how both processes work concurrently to support effective DDM.