Why more choices cause less cooperation in iterated prisoner's dilemma

The classic iterated prisoner's dilemma (IPD) has only 2 choices, cooperate or defect. However, most real-world situations offer intermediate responses, between full cooperation and full defection. Previous studies observed that with intermediate levels, mutual cooperation is less likely to emerge, and even if it does it is less stable. Exactly why has been a mystery. This paper demonstrates two mechanisms that sabotage the emergence of full mutual cooperation. First, to increase cooperation requires behavioral (phenotypic) diversity to explore different possible outcomes, and once evolution has converged somewhat on a particular degree of cooperation, it is unlikely to shift. Secondly, more choices allows a richer choice of stable strategies that are not simply cooperating with each other to exclude an invader, but which are symbiotic. Such non-symmetric and symbiotic players in the space of strategies act as roadblocks on the path to full cooperation.

[1]  John J. Grefenstette,et al.  A Coevolutionary Approach to Learning Sequential Decision Rules , 1995, ICGA.

[2]  G. Roberts,et al.  Development of cooperative relationships through increasing investment , 1998, Nature.

[3]  Xin Yao,et al.  Does extra genetic diversity maintain escalation in a co-evolutionary arms race , 2000 .

[4]  David B. Fogel,et al.  Evolution, neural networks, games, and intelligence , 1999, Proc. IEEE.

[5]  Kristian Lindgren,et al.  Evolutionary phenomena in simple dynamics , 1992 .

[6]  X. Yao,et al.  How important is your reputation in a multi-agent environment , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[7]  Xin Yao,et al.  On Evolving Robust Strategies for Iterated Prisoner's Dilemma , 1993, Evo Workshops.

[8]  Jordan B. Pollack,et al.  Co-Evolving Intertwined Spirals , 1996, Evolutionary Programming.

[9]  Roger Ware,et al.  Evolutionary stability in the repeated prisoner's dilemma , 1989 .

[10]  Daniel H. Janzen,et al.  How to be a Fig , 1979 .

[11]  Phil Husbands,et al.  Simulated Co-Evolution as the Mechanism for Emergent Planning and Scheduling , 1991, International Conference on Genetic Algorithms.

[12]  D. Fogel,et al.  Evolving continuous behaviors in the Iterated Prisoner's Dilemma. , 1996, Bio Systems.

[13]  Jeffrey W. Herrmann,et al.  A genetic algorithm for minimax optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[14]  Marcus Frean.  The evolution of degrees of cooperation. , 1996, Journal of theoretical biology.

[15]  R. Boyd,et al.  No pure strategy is evolutionarily stable in the repeated Prisoner's Dilemma game , 1987, Nature.

[16]  W. Hamilton,et al.  The evolution of cooperation. , 1984, Science.

[17]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[18]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[19]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[20]  Roland Schwaiger,et al.  Evolutionary and coevolutionary approaches to time series prediction using generalized multi-layer perceptrons , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[21]  Martin Braun.  Differential equations and their applications , 1976 .

[22]  Paul J. Darwen,et al.  Co-evolutionary learning on noisy tasks , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).