Control over patch encounters changes foraging behavior

Foraging is a common decision problem in natural environments. When new exploitable sites are always available, a simple optimal strategy is to leave a current site when its return falls below a single average reward rate. Here, we examined foraging in a more structured environment, with a limited number of sites that replenished at different rates and had to be revisited. When participants could choose sites, they visited fast-replenishing sites more often, left sites at higher levels of reward, and achieved a higher net reward rate. Decisions to exploit-or-leave a site were best explained with a computational model estimating separate reward rates for each site. This suggests option-specific information can be used to construct a threshold for patch leaving in some foraging settings, rather than a single average reward rate.

[1]  Timothy Edward John Behrens,et al.  Value, search, persistence and model updating in anterior cingulate cortex , 2016, Nature Neuroscience.

[2]  N. Kolling,et al.  Dopamine and motivational state drive dynamics of human decision making , 2019, bioRxiv.

[3]  H. Akaike.  A new look at the statistical model identification , 1974 .

[4]  Guy Cowlishaw,et al.  How do foragers decide when to leave a patch? A test of alternative models under natural and experimental conditions. , 2013, The Journal of animal ecology.

[5]  M. Eubanks,et al.  Overcompensation for insect herbivory: a review and meta-analysis of the evidence. , 2018, Ecology.

[6]  Z. Mainen,et al.  Activation of serotonin neurons promotes active persistence in a probabilistic foraging task , 2018, Nature Communications.

[7]  Steve W. C. Chang,et al.  Posterior Cingulate Neurons Dynamically Signal Decisions to Disengage during Foraging , 2017, Neuron.

[8]  Neil Garrett,et al.  Biased belief updating and suboptimal choice in foraging decisions , 2019, bioRxiv.

[9]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[10]  L. Chittka,et al.  Travel Optimization by Foraging Bumblebees through Readjustments of Traplines after Discovery of New Feeding Locations , 2010, The American Naturalist.

[11]  Timothy E. J. Behrens,et al.  Neural Mechanisms of Foraging , 2012, Science.

[12]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[13]  N. Daw,et al.  Chronic and Acute Stress Promote Overexploitation in Serial Decision Making , 2017, The Journal of Neuroscience.

[14]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[15]  R. Erwin,et al.  Foraging Decisions, Patch Use, and Seasonality in Egrets (Aves: Ciconiiformes) , 1985 .

[16]  John M. Pearson,et al.  Neuronal basis of sequential foraging decisions in a patchy environment , 2011, Nature Neuroscience.

[17]  Zeb Kurth-Nelson,et al.  Fast Sequences of Non-spatial State Representations in Humans , 2016, Neuron.

[18]  Jonathan R. Potts,et al.  Energy benefits and emergent space use patterns of an empirically parameterized model of memory‐based patch selection , 2017 .

[19]  A. Kamil.  Systematic foraging by a nectar-feeding bird, the Amakihi (Loxops virens) , 1978 .

[20]  E. Charnov.  Optimal foraging, the marginal value theorem. , 1976, Theoretical population biology.

[21]  M. Boyce,et al.  Patch-use dynamics by a large herbivore , 2015, Movement ecology.

[22]  F. Luyckx,et al.  Revisiting foraging approaches in neuroscience , 2019, Cognitive, Affective, & Behavioral Neuroscience.

[23]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[24]  Y. Niv.  Learning task-state representations , 2019, Nature Neuroscience.

[25]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[26]  Simon Benhamou,et al.  How Memory-Based Movement Leads to Nonterritorial Spatial Segregation , 2015, The American Naturalist.

[27]  Nathaniel D. Daw,et al.  Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task , 2017, Cognitive, Affective, & Behavioral Neuroscience.

[28]  C. Menzel,et al.  Memory and foraging theory: chimpanzee utilization of optimality heuristics in the rank-order recovery of hidden foods , 2012, Animal Behaviour.

[29]  Timothy E. J. Behrens,et al.  Review Frontal Cortex and Reward-guided Learning and Decision-making Figure 1. Frontal Brain Regions in the Macaque Involved in Reward-guided Learning and Decision-making Finer Grained Anatomical Divisions with Frontal Cortical Systems for Reward-guided Behavior , 2022 .

[30]  Zeb Kurth-Nelson,et al.  What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior , 2018, Neuron.

[31]  N. Daw,et al.  Learning the opportunity cost of time in a patch-foraging task , 2015, Cognitive, Affective, & Behavioral Neuroscience.

[32]  J M Hinson,et al.  Optimal choice. , 1981, Journal of the experimental analysis of behavior.

[33]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[34]  A. Houston,et al.  Optimal patch use by a territorial forager , 1990 .

[35]  J A Merkle,et al.  A memory-based foraging tactic reveals an adaptive mechanism for restricted space use. , 2014, Ecology letters.

[36]  Pete C. Trimmer,et al.  Foraging for foundations in decision neuroscience: insights from ethology , 2018, Nature Reviews Neuroscience.

[37]  Oded Berger-Tal,et al.  The Glass is Half-Full: Overestimating the Quality of a Novel Environment is Advantageous , 2012, PloS one.

[38]  R E Passingham.  Memory of monkeys (Macaca mulatta) with lesions in prefrontal cortex. , 1985, Behavioral neuroscience.

[39]  Brian M. Sweis,et al.  Mice learn to avoid regret , 2018, PLoS biology.

[40]  Bolton K. H. Chau,et al.  The macaque anterior cingulate cortex translates counterfactual choice value into actual behavioral change , 2018, Nature Neuroscience.

[41]  Alaa A. Ahmed,et al.  Control of movement vigor and decision making during foraging , 2018, Proceedings of the National Academy of Sciences.

[42]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[43]  Vaughn L. Hetrick,et al.  Mesolimbic Dopamine Signals the Value of Work , 2015, Nature Neuroscience.

[44]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[45]  J. Gray,et al.  PsychoPy2: Experiments in behavior made easy , 2019, Behavior Research Methods.

[46]  Nils Kolling,et al.  Predictive decision making driven by multiple time-linked reward representations in the anterior cingulate cortex , 2016, Nature Communications.