The role of uncertainty and reward on eye movements in a virtual driving task.

Eye movements during natural tasks are well coordinated with ongoing task demands and many variables could influence gaze strategies. Sprague and Ballard (2003) proposed a gaze-scheduling model that uses a utility-weighted uncertainty metric to prioritize fixations on task-relevant objects and predicted that human gaze should be influenced by both reward structure and task-relevant uncertainties. To test this conjecture, we tracked the eye movements of participants in a simulated driving task where uncertainty and implicit reward (via task priority) were varied. Participants were instructed to simultaneously perform a Follow Task where they followed a lead car at a specific distance and a Speed Task where they drove at an exact speed. We varied implicit reward by instructing the participants to emphasize one task over the other and varied uncertainty in the Speed Task with the presence or absence of uniform noise added to the car's velocity. Subjects' gaze data were classified for the image content near fixation and segmented into looks. Gaze measures, including look proportion, duration and interlook interval, showed that drivers more closely monitor the speedometer if it had a high level of uncertainty, but only if it was also associated with high task priority or implicit reward. The interaction observed appears to be an example of a simple mechanism whereby the reduction of visual uncertainty is gated by behavioral relevance. This lends qualitative support for the primary variables controlling gaze allocation proposed in the Sprague and Ballard model.

[1]  Journal of the Optical Society of America , 1950, Nature.

[2]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[3]  Charles L. Folk,et al.  The structure of attentional control: contingent attentional capture by apparent motion, abrupt onset, and color , 1994 .

[4]  R W Remington,et al.  The structure of attentional control: contingent attentional capture by apparent motion, abrupt onset, and color. , 1994, Journal of experimental psychology. Human perception and performance.

[5]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[6]  Michael Land,et al.  Which parts of the road guide steering? , 1995, Nature.

[7]  John R. Anderson ACT: A simple theory of complex cognition. , 1996 .

[8]  D. Ballard,et al.  Task constraints in visual working memory , 1997, Vision Research.

[9]  R. Remington,et al.  Selectivity in distraction by irrelevant featural singletons: evidence for two forms of attentional capture. , 1998, Journal of experimental psychology. Human perception and performance.

[10]  M. Land,et al.  The Roles of Vision and Eye Movements in the Control of Activities of Daily Living , 1998, Perception.

[11]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[12]  S. Yantis,et al.  On the distinction between visual salience and stimulus-driven attentional capture. , 1999, Journal of experimental psychology. Human perception and performance.

[13]  J. Schall,et al.  Performance monitoring by the supplementary eye ® eld , 2000 .

[14]  Andrew T. Duchowski,et al.  Proceedings of the 2006 symposium on Eye tracking research & applications , 2000 .

[15]  Andrew Y. Ng,et al.  Pharmacokinetics of a novel formulation of ivermectin after administration to goats , 2000, ICML.

[16]  M. Hayhoe,et al.  The coordination of eye, head, and hand movements in a natural task , 2001, Experimental Brain Research.

[17]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[18]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[19]  O. Hikosaka,et al.  Feature-Based Anticipation of Cues that Predict Reward in Monkey Caudate Nucleus , 2002, Neuron.

[20]  Dana H. Ballard,et al.  Eye Movements for Reward Maximization , 2003, NIPS.

[21]  O. Hikosaka,et al.  Neural Correlates of Rewarded and Unrewarded Eye Movements in the Primate Caudate Nucleus , 2003, The Journal of Neuroscience.

[22]  R. Jacobs,et al.  Experience-dependent visual cue recalibration based on discrepancies between visual and haptic percepts , 2003, Vision Research.

[23]  P. Glimcher The neurobiology of visual-saccadic decision making. , 2003, Annual review of neuroscience.

[24]  J. Wolfe,et al.  Changing your mind: on the contributions of top-down and bottom-up guidance in visual search for feature singletons. , 2003, Journal of experimental psychology. Human perception and performance.

[25]  P. Glimcher,et al.  Activity in Posterior Parietal Cortex Is Correlated with the Relative Subjective Desirability of Action , 2004, Neuron.

[26]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[27]  J. Theeuwes Top-down search strategies cannot override attentional capture , 2004, Psychonomic bulletin & review.

[28]  Rob Gray,et al.  A Two-Point Visual Control Model of Steering , 2004, Perception.

[29]  J. Wolfe,et al.  What attributes guide the deployment of visual attention and how do they do it? , 2004, Nature Reviews Neuroscience.

[30]  Jeff B. Pelz,et al.  Head movement estimation for wearable eye tracker , 2004, ETRA.

[31]  J. Maunsell Neuronal representations of cognitive state: reward or attention? , 2004, Trends in Cognitive Sciences.

[32]  D. Ballard,et al.  Eye movements in natural behavior , 2005, Trends in Cognitive Sciences.

[33]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[34]  L. Maloney,et al.  Explicit estimation of visual uncertainty in human motion processing , 2005, Vision Research.

[35]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[36]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[37]  Jason A. Droll,et al.  Task demands control acquisition and storage of visual information. , 2005, Journal of experimental psychology. Human perception and performance.

[38]  Michael L. Platt,et al.  Monkeys Pay Per View: Adaptive Valuation of Social Images by Rhesus Macaques , 2005, Current Biology.

[39]  Kae Nakamura,et al.  Basal ganglia orient eyes to reward. , 2006, Journal of neurophysiology.

[40]  Mary Hayhoe,et al.  Control of attention and gaze in complex environments. , 2006, Journal of vision.

[41]  Andrew B. Leber,et al.  It’s under control: Top-down search strategies can override attentional capture , 2006, Psychonomic bulletin & review.

[42]  J. Schall,et al.  Executive control of countermanding saccades by the supplementary eye field , 2006, Nature Neuroscience.

[43]  Daeyeol Lee,et al.  Effects of reward expectancy on sequential eye movements in monkeys , 2006, Neural Networks.

[44]  Preeti Verghese,et al.  Where to look next? Eye movements reduce local uncertainty. , 2007, Journal of vision.

[45]  Paul R. Schrater,et al.  Effects of visual uncertainty on grasping movements , 2007, Experimental Brain Research.

[46]  Dana H. Ballard,et al.  Modeling embodied visual behaviors , 2007, TAP.

[47]  Jillian H. Fecteau,et al.  Priming of pop-out depends upon the current goals of observers. , 2007, Journal of vision.

[48]  H. Seo,et al.  Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game , 2007, The Journal of Neuroscience.

[49]  John R. Anderson How Can the Human Mind Occur in the Physical Universe , 2007 .

[50]  E. Knudsen Fundamental components of attention. , 2007, Annual review of neuroscience.

[51]  George J. Andersen,et al.  Optical Information for Car Following: The Driving by Visual Angle (DVA) Model , 2007, Hum. Factors.

[52]  Julia Trommershäuser,et al.  Eye movements during rapid pointing under risk , 2007, Vision Research.

[53]  N. Lavie,et al.  Failures to Ignore Entirely Irrelevant Distractors , 2008, Journal of experimental psychology. Applied.

[54]  Dario D. Salvucci,et al.  Threaded cognition: an integrated theory of concurrent multitasking. , 2008, Psychological review.

[55]  John K. Tsotsos,et al.  Saliency, attention, and visual search: an information theoretic approach. , 2009, Journal of vision.

[56]  Peter A. Flach,et al.  Evaluation Measures for Multi-class Subgroup Discovery , 2009, ECML/PKDD.

[57]  Karl R Gegenfurtner,et al.  Effects of salience and reward information during saccadic decisions under risk. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[58]  M. Hayhoe,et al.  Adaptive Gaze Control in Natural Environments , 2009, The Journal of Neuroscience.

[59]  Pierre Baldi,et al.  Bayesian surprise attracts human attention , 2005, Vision Research.

[60]  Pietro Perona,et al.  Homo economicus in visual search. , 2009, Journal of vision.

[61]  Ethan S. Bromberg-Martin,et al.  Midbrain Dopamine Neurons Signal Preference for Advance Information about Upcoming Rewards , 2009, Neuron.

[62]  Christopher J. Peck,et al.  Reward Modulates Attention Independently of Action Value in Posterior Parietal Cortex , 2009, The Journal of Neuroscience.

[63]  Shihui Han,et al.  Attentional capture is contingent on the interaction between task demand and stimulus salience , 2009, Attention, perception & psychophysics.

[64]  Matthew H Tong,et al.  SUN: Top-down saliency using natural statistics , 2009, Visual cognition.

[65]  Dana H. Ballard,et al.  Credit Assignment in Multiple Goal Embodied Visuomotor Behavior , 2010, Front. Psychology.

[66]  Puiu F. Balan,et al.  Attention as a decision in information space , 2010, Trends in Cognitive Sciences.

[67]  Pietro Perona,et al.  Optimal reward harvesting in complex perceptual environments , 2010, Proceedings of the National Academy of Sciences.

[68]  Hang Zhang,et al.  Gambling in the Visual Periphery: A Conjoint-Measurement Analysis of Human Ability to Judge Visual Uncertainty , 2010, PLoS Comput. Biol..

[69]  Christos Dimitrakakis,et al.  Preference elicitation and inverse reinforcement learning , 2011, ECML/PKDD.

[70]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[71]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[72]  Mary Hayhoe,et al.  A modular reinforcement learning model for human visuomotor behavior in a driving task , 2011 .

[73]  Niels Taatgen,et al.  Toward a Unified View of Cognitive Control , 2011, Top. Cogn. Sci..

[74]  Laurence T. Maloney,et al.  Human Visual Search Does Not Maximize the Post-Saccadic Probability of Identifying Targets , 2012, PLoS Comput. Biol..

[75]  J. Wickens,et al.  Neural control of dopamine neurotransmission: implications for reinforcement learning , 2012, The European journal of neuroscience.

[76]  Erich W Graf,et al.  Visual extrapolation under risk: human observers estimate and compensate for exogenous uncertainty , 2012, Proceedings of the Royal Society B: Biological Sciences.

[77]  Alexander C. Schütz,et al.  Dynamic integration of information about salience and value for saccadic eye movements , 2012, Proceedings of the National Academy of Sciences.

[78]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[79]  Balaraman Ravindran,et al.  Where do i look now? Gaze allocation during visually guided manipulation , 2012, 2012 IEEE International Conference on Robotics and Automation.

[80]  Duncan P. Brumby,et al.  Natural Break Points , 2012 .

[81]  Dana H. Ballard,et al.  Modular inverse reinforcement learning for visuomotor behavior , 2013, Biological Cybernetics.