Individually irrational pruning is essential for ecological rationality in a social context

Heuristics, commonly thought to violate the full rationality assumptions, are paradoxically indispensable parts of our decision-making and learning processes. To resolve this seemingly paradox, there have been several studies in the literature that aim at finding some broad daily life conditions and situations where employing heuristics are rational. However, these researches mainly focus on non-social conditions, whereas, for human beings, social and individual processes are interwoven and it would be better to study them jointly. Here, we study the role of pruning heuristic in individual reinforcement learning in a social context, where our simulated learning agents make many of their decisions relying on others' knowledge. Our simulation results suggest that the seemingly irrational pruning heuristic leads to less cost in the social settings. That is, we have a meaningfully more social outcome in the presence of this heuristic in social contexts, and social learning helps the agents to learn better where the pruning heuristic is an obstacle in the way of finding the optimal solution in the individual setting. In sum, the synergy between the pruning behavior and social learning leads to ecological rationality.

[1]  Iain D. Couzin,et al.  Collective Learning and Optimal Consensus Decisions in Social Animal Groups , 2014, PLoS Comput. Biol..

[2]  Richard McElreath,et al.  The evolutionary rationality of social learning , 2013 .

[3]  Ulrich Hoffrage,et al.  Simple heuristics in a social world , 2012 .

[4]  Eric J. Johnson,et al.  The adaptive decision maker , 1993 .

[5]  Wolfgang M. Pauli,et al.  Learning, Reward, and Decision Making , 2017, Annual review of psychology.

[6]  P. Todd,et al.  Simple Heuristics That Make Us Smart , 1999 .

[7]  Peter Dayan,et al.  Serotonin, Inhibition, and Negative Mood , 2007, PLoS Comput. Biol..

[8]  Peter M. Todd,et al.  Building the Theory of Ecological Rationality , 2015, Minds and Machines.

[9]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[10]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[11]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[12]  Tim Johnson,et al.  Cooperate with Equals: A Simple Heuristic for Social Exchange , 2012 .

[13]  L. Giraldeau,et al.  Exploring the costs and benefits of social information use: an appraisal of current experimental evidence , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[14]  P. Dayan,et al.  Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation , 2014, Cognitive, affective & behavioral neuroscience.

[15]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[16]  David P. Myatt,et al.  Innovation Diffusion in Heterogeneous Populations: Contagion, Social Influence, and Social Learning , 2009 .

[17]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[18]  Peter Dayan,et al.  Rationalizable Irrationalities of Choice , 2014, Top. Cogn. Sci..

[19]  Lael J. Schooler,et al.  When Will We Meet Again? Regularities of Social Connectivity and Their Reflections in Memory and Decision Making , 2013 .

[20]  T. Griffiths,et al.  Strategy Selection as Rational Metareasoning , 2017, Psychological review.

[21]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[22]  T. Lillicrap,et al.  Why Copy Others? Insights from the Social Learning Strategies Tournament , 2010, Science.

[23]  Allen Newell,et al.  Human problem solving: The state of the theory in 1970. , 1971 .

[24]  Gerd Gigerenzer,et al.  Heuristic decision making. , 2011, Annual review of psychology.

[25]  U. Hoffrage,et al.  Ecological Rationality for Teams and CommitteesHeuristics in Group Decision Making , 2012 .

[26]  Shinsuke Shimojo,et al.  Neural Computations Underlying Arbitration between Model-Based and Model-free Learning , 2013, Neuron.

[27]  Wouter Kool,et al.  When Does Model-Based Control Pay Off? , 2016, PLoS Comput. Biol..

[28]  J. Rieskamp,et al.  How social information affects information search and choice in probabilistic inferences. , 2018, Acta psychologica.

[29]  N. Harvey,et al.  Taking Advice: Accepting Help, Improving Judgment, and Sharing Responsibility☆☆☆ , 1997 .

[30]  A. Whiten,et al.  The multiple roles of cultural transmission experiments in understanding human cultural evolution , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[31]  Thomas L. Griffiths,et al.  Cognitive prostheses for goal achievement , 2019, Nature Human Behaviour.

[32]  Gerd Gigerenzer,et al.  Social learning: A route to good cue orders , 2013 .

[33]  Mark Lubell,et al.  Beyond existence and aiming outside the laboratory: estimating frequency-dependent and pay-off-biased social learning strategies , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[34]  Yael Niv,et al.  How to divide and conquer the world, one step at a time , 2015, Proceedings of the National Academy of Sciences.

[35]  Falk Lieder,et al.  Overrepresentation of Extreme Events in Decision Making Reflects Rational Use of Cognitive Resources , 2017, Psychological review.

[36]  R. Boyd,et al.  A Bayesian approach to the evolution of social learning , 2012 .

[37]  K. Laland,et al.  Cognitive culture: theoretical and empirical insights into social learning strategies , 2011, Trends in Cognitive Sciences.

[38]  R. Mcelreath,et al.  STRATEGIES FOR ADVICE TAKING: THE ROLE OF EPISTEMIC SOCIAL INFORMATION , 2015, Economics and Philosophy.

[39]  P. Dayan,et al.  Reinforcement learning: The Good, The Bad and The Ugly , 2008, Current Opinion in Neurobiology.

[40]  G. Gigerenzer,et al.  Does imitation benefit cue order learning? , 2009, Experimental psychology.

[41]  Peter Bossaerts,et al.  Evidence for Model-based Computations in the Human Amygdala during Pavlovian Conditioning , 2013, PLoS Comput. Biol..

[42]  Jan K. Woike,et al.  Social contact patterns can buffer costs of forgetting in the evolution of cooperation , 2018, Proceedings of the Royal Society B: Biological Sciences.

[43]  Thorsten Pachur,et al.  Intuitive judgments of social statistics : How exhaustive does sampling need to be? , 2013 .

[44]  Daniel M. Oppenheimer,et al.  Heuristics made easy: an effort-reduction framework. , 2008, Psychological bulletin.

[45]  C. Heyes.  Who Knows? Metacognitive Social Learning Strategies , 2016, Trends in Cognitive Sciences.

[46]  L. Schooler,et al.  We'll Meet Again: Revealing Distributional and Temporal Patterns of Social Contact , 2014, PloS one.

[47]  K. Laland.  Social learning strategies , 2004, Learning & behavior.

[48]  R. Hertwig,et al.  Heuristics: The Foundations of Adaptive Behavior , 2015 .

[49]  Robin M. Hogarth,et al.  Regions of Rationality: Maps for Bounded Agents , 2005, Decis. Anal..

[50]  Amir Dezfouli,et al.  Speed/Accuracy Trade-Off between the Habitual and the Goal-Directed Processes , 2011, PLoS Comput. Biol..

[51]  Tom Wenseleers,et al.  Uncertainty about social interactions leads to the evolution of social heuristics , 2018, Nature Communications.

[52]  H. Simon,et al.  Invariants of human behavior. , 1990, Annual review of psychology.

[53]  Ulrich Hoffrage,et al.  Simple heuristics: The foundations of adaptive social behavior , 2013 .

[54]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[55]  Ilan Yaniv,et al.  Receiving Other People's Advice: Influence and Benefit , 2004 .