A model of risk and mental state shifts during social interaction

Cooperation and competition between human players in repeated microeconomic games offer a window onto social phenomena such as the establishment, breakdown and repair of trust. However, although a suitable starting point for the quantitative analysis of such games exists, namely the Interactive Partially Observable Markov Decision Process (I-POMDP), computational considerations and structural limitations have limited its application, and left unmodelled critical features of behavior in a canonical trust task. Here, we provide the first analysis of two central phenomena: a form of social risk-aversion exhibited by the player who is in control of the interaction in the game; and irritation or anger, potentially exhibited by both players. Irritation arises when partners apparently defect, and it potentially causes a precipitate breakdown in cooperation. Failing to model one’s partner’s propensity for it leads to substantial economic inefficiency. We illustrate these behaviours using evidence drawn from the play of large cohorts of healthy volunteers and patients. We show that for both cohorts, a particular subtype of player is largely responsible for the breakdown of trust, a finding which sheds new light on borderline personality disorder.

[1]  John P Elder,et al.  Interpersonal and print nutrition communication for a Spanish-dominant Latino population: Secretos de la Buena Vida. , 2005, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[2]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[3]  Prashant Doshi,et al.  Modeling recursive reasoning by humans using empirically informed interactive POMDPs , 2010, AAMAS.

[4]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[5]  Raymond J. Dolan,et al.  Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding , 2011, PLoS Comput. Biol..

[6]  P. J. Gmytrasiewicz,et al.  A Framework for Sequential Planning in Multi-Agent Settings , 2005, AI&M.

[7]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[8]  J. Harsanyi.  Games with Incomplete Information Played by 'Bayesian' Players, Part III. The Basic Probability Distribution of the Game , 1968 .

[9]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[10]  Miguel A. Costa-Gomes,et al.  Cognition and Behavior in Normal-Form Games: An Experimental Study , 1998 .

[11]  Robert J . Aumann,et al.  28. Mixed and Behavior Strategies in Infinite Extensive Games , 1964 .

[12]  T. Lohrenz,et al.  Computational Substrates of Norms and Their Violations during Social Exchange , 2013, The Journal of Neuroscience.

[13]  Peter Auer,et al.  The Nonstochastic Multiarmed Bandit Problem , 2002, SIAM J. Comput..

[14]  Daniel Houser,et al.  A functional imaging study of cooperation in two-person reciprocal exchange , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[15]  R. McKelvey,et al.  Quantal Response Equilibria for Extensive Form Games , 1998 .

[16]  Peter Dayan,et al.  Bayesian Model of Behaviour in Economic Games , 2008, NIPS.

[17]  M. Brüne,et al.  Borderline Personality Disorder , 2016, Evolution, medicine, and public health.

[18]  Pearl H. Chiu,et al.  Self Responses along Cingulate Cortex Reveal Quantitative Neural Phenotype for High-Functioning Autism , 2008, Neuron.

[19]  J. Harsanyi.  Games with randomly disturbed payoffs: A new rationale for mixed-strategy equilibrium points , 1973 .

[20]  E. Fehr.  A Theory of Fairness, Competition and Cooperation , 1998 .

[21]  Prashant Doshi,et al.  Monte Carlo Sampling Methods for Approximating Interactive POMDPs , 2014, J. Artif. Intell. Res..

[22]  Colin Camerer,et al.  A Cognitive Hierarchy Model of Games , 2004 .

[23]  Klaus Lieb,et al.  Borderline personality disorder , 2004, The Lancet.

[24]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[25]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[26]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[27]  S. Quartz,et al.  Getting to Know You: Reputation and Trust in a Two-Person Economic Exchange , 2005, Science.

[28]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[29]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Joachim Winter,et al.  Distinguishing Trust from Risk: An Anatomy of the Investment Game , 2009 .

[31]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[32]  Michael L. Littman,et al.  Using iterated reasoning to predict opponent strategies , 2011, AAMAS.

[33]  John C. Harsanyi,et al.  Games with Incomplete Information Played by "Bayesian" Players, I-III: Part I. The Basic Model& , 2004, Manag. Sci..

[34]  Chris L. Baker,et al.  Rational quantitative attribution of beliefs, desires and percepts in human mentalizing , 2017, Nature Human Behaviour.

[35]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[36]  Joyce E. Berg,et al.  Trust, Reciprocity, and Social History , 1995 .

[37]  G. Loewenstein,et al.  Hot-cold empathy gaps and medical decision making. , 2005, Health psychology : official journal of the Division of Health Psychology, American Psychological Association.

[38]  Peter Fonagy,et al.  Mechanisms of change in mentalization-based treatment of BPD. , 2006, Journal of clinical psychology.

[39]  Prashant Doshi,et al.  Modeling Human Recursive Reasoning Using Empirically Informed Interactive Partially Observable Markov Decision Processes , 2012, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[40]  Marina Vannucci,et al.  Biosensor Approach to Psychopathology Classification , 2010, PLoS Comput. Biol..

[41]  Prashant Doshi,et al.  Modeling Deep Strategic Reasoning by Humans in Competitive Games , 2012, ISAIM.

[42]  Peter Dayan,et al.  Monte Carlo Planning Method Estimates Planning Horizons during Interactive Social Exchange , 2015, PLoS Comput. Biol..

[43]  Colin Camerer.  Behavioral Game Theory: Experiments in Strategic Interaction , 2003 .

[44]  R. McKelvey,et al.  An experimental study of the centipede game , 1992 .

[45]  B. King-Casas,et al.  The Rupture and Repair of Cooperation in Borderline Personality Disorder , 2008, Science.

[46]  Peter Dayan,et al.  Computational Phenotyping of Two-Person Interactions Reveals Differential Neural Response to Depth-of-Thought , 2012, PLoS Comput. Biol..

[47]  V. Smith,et al.  Positive reciprocity and intentions in trust games , 2003 .