History mechanism supported differential evolution for chess evaluation function tuning

This paper presents a differential evolution (DE) based approach to chess evaluation function tuning. DE with opposition-based optimization is employed and upgraded with a history mechanism to improve the evaluation of individuals and the tuning process. The general idea is based on individual evaluations according to played games through several generations and different environments. We introduce a new history mechanism which uses an auxiliary population containing good individuals. This new mechanism ensures that good individuals remain within the evolutionary process, even though they died several generations back and later can be brought back into the evolutionary process. In such a manner the evaluation of individuals is improved and consequently the whole tuning process.

[1]  Andrew Tridgell,et al.  Experiments in Parameter Learning Using Temporal Differences , 1998, J. Int. Comput. Games Assoc..

[2]  R. Storn,et al.  Differential evolution a simple and efficient adaptive scheme for global optimization over continu , 1997 .

[3]  Ernst A. Heinz.  Scalable search in computer chess: algorithmic enhancements and experiments at high search depths , 1999 .

[4]  Robert Levinson,et al.  Adaptive Pattern-Oriented Chess , 1991, AAAI Conference on Artificial Intelligence.

[5]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[6]  Thomas S. Anantharaman,et al.  Evaluation Tuning for Computer Chess: Linear Discriminant Methods , 1997, J. Int. Comput. Games Assoc..

[7]  Michael Buro,et al.  Tuning evaluation functions by maximizing concordance , 2005, Theor. Comput. Sci..

[8]  Sebastian Thrun,et al.  Learning to Play the Game of Chess , 1994, NIPS.

[9]  Gualtiero Piccinini,et al.  Alan Turing and the Mathematical Objection , 2003, Minds and Machines.

[10]  Dexian Huang,et al.  Multi-objective no-wait flow-shop scheduling with a memetic algorithm based on differential evolution , 2009, Soft Comput..

[11]  Janez Brest,et al.  Performance comparison of self-adaptive and adaptive differential evolution algorithms , 2007, Soft Comput..

[12]  Thomas S. Anantharaman,et al.  A Statistical Study of Selective Min-Max Search in Computer Chess , 1991, J. Int. Comput. Games Assoc..

[13]  Ville Tirronen,et al.  Super-fit control adaptation in memetic differential evolution frameworks , 2009, Soft Comput..

[14]  Frank Y. Shih,et al.  A differential evolution based algorithm for breaking the visual steganalytic system , 2008, Soft Comput..

[15]  D.B. Fogel,et al.  A self-learning evolutionary chess program , 2004, Proceedings of the IEEE.

[16]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[17]  Vitaliy Feoktistov,et al.  Differential Evolution: In Search of Solutions (Springer Optimization and Its Applications) , 2006 .

[18]  Jason Teo,et al.  Self-adaptive population sizing for a tune-free differential evolution , 2009, Soft Comput..

[19]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[20]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[21]  David B. Fogel,et al.  Evolving neural networks to play checkers without relying on expert knowledge , 1999, IEEE Trans. Neural Networks.

[22]  Graham Kendall,et al.  An evolutionary approach for the tuning of a chess evaluation function using population dynamics , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[23]  Donald F. Beal,et al.  Learning Piece Values Using Temporal Differences , 1997, J. Int. Comput. Games Assoc..

[24]  Donald F. Beal,et al.  Learning Piece-square Values using Temporal Differences , 1999, J. Int. Comput. Games Assoc..

[25]  Janez Brest,et al.  A Differential Evolution for the Tuning of a Chess Evaluation Function , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[26]  Janez Brest,et al.  An Adaptive Differential Evolution Algorithm with Opposition-Based Mechanisms, Applied to the Tuning of a Chess Program , 2008 .

[27]  D. Hunter.  MM algorithms for generalized Bradley-Terry models , 2003 .

[28]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[29]  Andrew Tridgell,et al.  Learning to Play Chess Using Temporal Differences , 2000, Machine Learning.

[30]  Gerald Tesauro,et al.  Comparison training of chess evaluation functions , 2001 .

[31]  Johannes Fürnkranz,et al.  Machine learning in games: a survey , 2001 .

[32]  A. L. Samuel,et al.  Some studies in machine learning using the game of checkers. II: recent progress , 1967 .

[33]  Arthur L. Samuel,et al.  Some studies in machine learning using the game of checkers , 2000, IBM J. Res. Dev..

[34]  David Clark.  Deep thoughts on Deep Blue , 1997 .

[35]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[36]  Claude E. Shannon,et al.  Programming a computer for playing chess , 1950 .

[37]  Hamid R. Tizhoosh,et al.  Opposition-Based Learning: A New Scheme for Machine Intelligence , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).

[38]  H. Nasreddine,et al.  Using an Evolutionary Algorithm for the Tuning of a Chess Evaluation Function Based on a Dynamic Boundary Strategy , 2006, 2006 IEEE Conference on Cybernetics and Intelligent Systems.

[39]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[40]  Claude E. Shannon,et al.  XXII. Programming a Computer for Playing Chess 1 , 1950 .

[41]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[42]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[43]  David B. Fogel,et al.  The Blondie25 Chess Program Competes Against Fritz 8.0 and a Human Chess Master , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[44]  Janez Brest,et al.  Population size reduction for the differential evolution algorithm , 2008, Applied Intelligence.

[45]  Jason Teo,et al.  Exploring dynamic self-adaptive populations in differential evolution , 2006, Soft Comput..