Dynamic Integration of Reward and Stimulus Information in Perceptual Decision-Making

In perceptual decision-making, ideal decision-makers should bias their choices toward alternatives associated with larger rewards, and the extent of the bias should decrease as stimulus sensitivity increases. When responses must be made at different times after stimulus onset, stimulus sensitivity grows with time from zero to a final asymptotic level. Are decision makers able to produce responses that are more biased if they are made soon after stimulus onset, but less biased if they are made after more evidence has been accumulated? If so, how close to optimal can they come in doing this, and how might their performance be achieved mechanistically? We report an experiment in which the payoff for each alternative is indicated before stimulus onset. Processing time is controlled by a “go” cue occurring at different times post stimulus onset, requiring a response within msec. Reward bias does start high when processing time is short and decreases as sensitivity increases, leveling off at a non-zero value. However, the degree of bias is sub-optimal for shorter processing times. We present a mechanistic account of participants' performance within the framework of the leaky competing accumulator model [1], in which accumulators for each alternative accumulate noisy information subject to leakage and mutual inhibition. The leveling off of accuracy is attributed to mutual inhibition between the accumulators, allowing the accumulator that gathers the most evidence early in a trial to suppress the alternative. Three ways reward might affect decision making in this framework are considered. One of the three, in which reward affects the starting point of the evidence accumulation process, is consistent with the qualitative pattern of the observed reward bias effect, while the other two are not. Incorporating this assumption into the leaky competing accumulator model, we are able to provide close quantitative fits to individual participant data.

[1]  W. Edwards.  Optimal strategies for seeking information: Models for statistics, choice reaction times, and human information processing ☆ , 1965 .

[2]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[3]  C. C. Wood.  Discriminability, response bias, and phoneme categories in discrimination of voice onset time. , 1976, The Journal of the Acoustical Society of America.

[4]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[5]  James T. Townsend.  Some characteristics of visual whole report behavior , 1981 .

[6]  T. Dusoir,et al.  Isobias curves in some detection tasks , 1983, Perception & psychophysics.

[7]  J. Busemeyer.  Decision making under uncertainty: a comparison of simple scalability, fixed-sample, and sequential-sampling models. , 1985, Journal of experimental psychology. Learning, memory, and cognition.

[8]  Jerome R. Busemeyer,et al.  Psychological models of deferred decision making , 1988 .

[9]  Elke U. Weber,et al.  Expectation and variance of item resemblance distributions in a convolution-correction model of distributed memory , 1988 .

[10]  N. Macmillan,et al.  Response bias : characteristics of detection theory, threshold theory, and nonparametric indexes , 1990 .

[11]  G. Loftus,et al.  Sensory and cognitive components of visual information acquisition. , 1994, Psychological review.

[12]  J. Murray,et al.  Imagining and naming rotated natural objects , 1995, Psychonomic bulletin & review.

[13]  A. Kacelnik,et al.  Risk-sensitivity: crossroads for theories of decision-making , 1997, Trends in Cognitive Sciences.

[14]  B. Alsop.  Receiver operating characteristics from nonhuman animals: Some implications and directions for research with humans , 1998 .

[15]  R. Ratcliff,et al.  Connectionist and diffusion models of reaction time. , 1999, Psychological review.

[16]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[17]  Peter Dayan,et al.  Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems , 2001 .

[18]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[19]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[20]  Corey J Bohil,et al.  A theoretical framework for understanding the effects of simultaneous base-rate and payoff manipulations on decision criterion learning in perceptual categorization. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[21]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[22]  P. Glimcher,et al.  Neuroeconomics: The Consilience of Brain and Decision , 2004, Science.

[23]  Philip Holmes,et al.  Simple Neural Networks that Optimize Decisions , 2005, Int. J. Bifurc. Chaos.

[24]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[25]  R. Ratcliff.  Modeling response signal and response time data , 2006, Cognitive Psychology.

[26]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[27]  A. Diederich,et al.  Modeling the effects of payoff on response bias in a perceptual discrimination task: Bound-change, drift-rate-change, or two-stage-processing hypothesis , 2006, Perception & psychophysics.

[28]  Timothy D. Hanks,et al.  Bounded Integration in Parietal Cortex Underlies Decisions Even When Viewing Duration Is Dictated by the Environment , 2008, The Journal of Neuroscience.

[29]  J I Gold,et al.  On diffusion processes with variable drift rates as models for decision making during learning , 2008, New journal of physics.

[30]  Adele Diederich,et al.  A further test of sequential-sampling models that account for payoff effects on response bias in perceptual decision tasks , 2008, Perception & psychophysics.

[31]  KongFatt Wong-Lin,et al.  Sequential Effects in Two-Choice Reaction Time Tasks: Decomposition and Synthesis of Mechanisms , 2009, Neural Computation.

[32]  Jonathan D. Cohen,et al.  Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. , 2009, Journal of experimental psychology. Human perception and performance.

[33]  Philip Holmes,et al.  Can Monkeys Choose Optimally When Faced with Noisy Stimuli and Unequal Rewards? , 2009, PLoS Comput. Biol..

[34]  James L. McClelland,et al.  Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey , 2010, PloS one.