Interactive Learning of Top-down Attention Control and Motor Actions

Like humans and primates, artificial creatures like robots are limited in terms of allocation of their resources to huge sensory and perceptual information. Thus attention is regarded as the same solution as humans in this domain. While bottom-up attention is determined by the image statistics, top down attention is dependent the on behavior and the task an agent is doing. This work attempts to consider a task based top-down visual attention control when resources of the agent are limited. Particularly attention control is formulated as an optimization problem in which the agent has to gain maximum reward while satisfying a constraint which is its information processing bottleneck. Reinforcement learning is then used to solve that optimization problem. A driving environment is simulated in that agent has to learn how to drive safely by attending to the right spatial locations and performing appropriate motor actions.

[1]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[2]  Lucas Paletta,et al.  Cascaded Sequential Attention for Object Recognition with Informative Local Descriptors and Q-learning of Grouping Strategies , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[3]  Horst Bischof,et al.  Attentive Object Detection Using an Information Theoretic Saliency Measure , 2004, WAPCV.

[4]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[5]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[6]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[7]  U. Rieder,et al.  Markov Decision Processes , 2010 .

[8]  Andy Clark,et al.  Towards a Cognitive Robotics , 1999, Adapt. Behav..

[9]  A. L. Yarbus.  Eye Movements During Perception of Complex Objects , 1967 .

[10]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[11]  Dana H. Ballard,et al.  Modeling embodied visual behaviors , 2007, TAP.

[12]  Michael C. Mozer,et al.  Top-Down Control of Visual Attention: A Rational Account , 2005, NIPS.

[13]  Sridhar Mahadevan,et al.  A reinforcement learning model of selective visual attention , 2001, AGENTS '01.

[14]  Erik D. Reichle,et al.  Using reinforcement learning to understand the emergence of "intelligent" eye-movement behavior during reading. , 2006, Psychological review.

[15]  Laurent Itti,et al.  Applying computational tools to predict gaze direction in interactive visual environments , 2008, TAP.

[16]  Gilson A. Giraldi,et al.  Learning policies for attentional control , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[17]  Majid Nili Ahmadabadi,et al.  Biologically Inspired Framework for Learning and Abstract Representation of Attention Control , 2008, WAPCV.