Behavioral diversity, choices and noise in the iterated prisoner's dilemma

Real-world dilemmas rarely involve just two choices and perfect interactions without mistakes. In the iterated prisoner's dilemma (IPD) game, intermediate choices or mistakes (noise) have been introduced to extend its realism. This paper studies the IPD game with both noise and multiple levels of cooperation (intermediate choices) in a coevolutionary environment, where players can learn and adapt their strategies through an evolutionary algorithm. The impact of noise on the evolution of cooperation is first examined. It is shown that the coevolutionary models presented in this paper are robust against low noise (when mistakes occur with low probability). That is, low levels of noise have little impact on the evolution of cooperation. On the other hand, high noise (when mistakes occur with high probability) creates misunderstandings and discourages cooperation. However, the evolution of cooperation in the IPD with more choices in a coevolutionary learning setting also depends on behavioral diversity. This paper further investigates the issue of behavioral diversity in the coevolution of strategies for the IPD with more choices and noise. The evolution of cooperation is more difficult to achieve if a coevolutionary model with low behavioral diversity is used for IPD games with higher levels of noise. The coevolutionary model with high behavioral diversity in the population is more resistant to noise. It is shown that strategy representations can have a significant impact on the evolutionary outcomes because of different behavioral diversities that they generate. The results further show the importance of behavioral diversity in coevolutionary learning.

[1]  R. Schiffer,et al.  INTRODUCTION , 1988, Neurology.

[2]  R. Axelrod.  More Effective Choice in the Prisoner's Dilemma , 1980 .

[3]  Daniel B. Neill,et al.  Optimality under noise: higher memory strategies for the alternating prisoner's dilemma. , 2001, Journal of theoretical biology.

[4]  Wirt Atmar,et al.  Notes on the simulation of evolution , 1994, IEEE Trans. Neural Networks.

[5]  P. Molander.  The Optimal Level of Generosity in a Selfish, Uncertain Environment , 1985 .

[6]  P Kitcher,et al.  Evolution of altruism in optional and compulsory games. , 1995, Journal of theoretical biology.

[7]  Xin Yao,et al.  Does extra genetic diversity maintain escalation in a co-evolutionary arms race , 2000 .

[8]  L M Wahl,et al.  The Continuous Prisoner:s Dilemma: I. Linear Reactive Strategies , 1999 .

[9]  Xin Yao,et al.  On Evolving Robust Strategies for Iterated Prisoner's Dilemma , 1993, Evo Workshops.

[10]  R. Axelrod.  Effective Choice in the Prisoner's Dilemma , 1980 .

[11]  Bryant A. Julstrom.  Effects of Contest Length and Noise on Reciprocal Altruism, Cooperation, and Payoffs in the Iterated Prisoner's Dilemma , 1997, ICGA.

[12]  Mark D. Smucker,et al.  Iterated Prisoner's Dilemma with Choice and Refusal of Partners: Evolutionary Results , 1995, ECAL.

[13]  Roderick M. Kramer,et al.  When in Doubt... , 1991 .

[14]  X. Yao,et al.  How important is your reputation in a multi-agent environment , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[15]  W. Hamilton,et al.  The Evolution of Cooperation , 1984 .

[16]  M. Nowak,et al.  The continuous Prisoner's dilemma: II. Linear reactive strategies with noise. , 1999, Journal of theoretical biology.

[17]  David B. Fogel,et al.  On the Relationship between the Duration of an Encounter and the Evolution of Cooperation in the Iterated Prisoner's Dilemma , 1995, Evolutionary Computation.

[18]  R. Axelrod,et al.  How to Cope with Noise in the Iterated Prisoner's Dilemma , 1995 .

[19]  Xin Yao,et al.  Co-Evolution in Iterated Prisoner's Dilemma with Intermediate Levels of Cooperation: Application to Missile Defense , 2002, Int. J. Comput. Intell. Appl..

[20]  J. Bendor,et al.  Effective Choice in the Prisoner ' s Dilemma , 2007 .

[21]  Xin Yao,et al.  Why more choices cause less cooperation in iterated prisoner's dilemma , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[22]  D. Fogel,et al.  Evolving continuous behaviors in the Iterated Prisoner's Dilemma. , 1996, Bio Systems.

[23]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[24]  Robert Axelrod,et al.  The Evolution of Strategies in the Iterated Prisoner's Dilemma , 2001 .

[25]  R. Axelrod,et al.  The Further Evolution of Cooperation , 1988, Science.

[26]  David B. Fogel,et al.  Evolving Behaviors in the Iterated Prisoner's Dilemma , 1993, Evolutionary Computation.

[27]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[28]  D. Fogel.  The evolution of intelligent decision making in gaming , 1991 .

[29]  David B. Fogel,et al.  An introduction to simulated evolutionary optimization , 1994, IEEE Trans. Neural Networks.

[30]  Barbara Sainty,et al.  Achieving greater cooperation in a noisy prisoner’s dilemma: an experimental investigation ☆ , 1999 .

[31]  Alasdair I. Houston,et al.  Variation in behaviour promotes cooperation in the Prisoner's Dilemma game , 2004, Nature.

[32]  Xin Yao,et al.  The impact of noise on iterated prisoner's dilemma with multiple levels of cooperation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).