Strong, Stable, and Reliable Fitness Pressure in XCS due to Tournament Selection

Recent analysis of the XCS classifier system have shown that successful genetic learning strongly depends on the amount of fitness pressure towards accurate classifiers. Since the traditionally used proportionate selection is dependent on fitness scaling and fitness distribution, the resulting evolutionary fitness pressure may be neither stable nor sufficiently strong. Thus, we apply tournament selection to XCS. In particular, we exhibit the weakness of proportionate selection and suggest tournament selection as a more reliable alternative. We show that tournament selection results in a learning classifier system that is more parameter independent, noise independent, and more efficient in exploiting fitness guidance in single-step problems as well as multistep problems. The evolving population is more focused on promising subregions of the problem space and thus finds the desired accurate, maximally general representation faster and more reliably.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[3]  C. Watkins Learning from delayed rewards , 1989 .

[4]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[5]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[6]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[7]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[8]  Dirk Thierens,et al.  Mixing in Genetic Algorithms , 1993, ICGA.

[9]  Gilles Venturini,et al.  Adaptation in dynamic environments through a minimal probability of exploration , 1994 .

[10]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[11]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[12]  G. Harik Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms , 1997 .

[13]  Stewart W. Wilson Generalization in the XCS Classifier System , 1998 .

[14]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[15]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[16]  M. Colombetti,et al.  An extension to the XCS classifier system for stochastic environments , 1999 .

[17]  Martin V. Butz,et al.  An Algorithmic Description of XCS , 2000, IWLCS.

[18]  Stewart W. Wilson,et al.  Toward Optimal Classifier System Performance in Non-Markov Environments , 2000, Evolutionary Computation.

[19]  Tim Kovacs,et al.  What Makes a Problem Hard for XCS? , 2000, IWLCS.

[20]  P. Lanzi,et al.  Adaptive Agents with Reinforcement Learning and Internal Memory , 2000 .

[21]  D. Goldberg,et al.  A practical schema theorem for genetic algorithm design and tuning , 2001 .

[22]  Xavier Llorà,et al.  XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining , 2001, IWLCS.

[23]  Martin J. Oates,et al.  A Preliminary Investigation of Modified XCS as a Generic Data Mining Tool , 2001, IWLCS.

[24]  Martin V. Butz,et al.  How XCS evolves accurate classifiers , 2001 .

[25]  M. Pelikán,et al.  Analyzing the evolutionary pressures in XCS , 2001 .

[26]  Tim Kovacs,et al.  XCS's Strength-Based Twin: Part II , 2002, IWLCS.

[27]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[28]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[29]  Pier Luca Lanzi Learning classifier systems from a reinforcement learning perspective , 2002, Soft Comput..

[30]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[31]  Tim Kovacs,et al.  XCS's Strength-Based Twin: Part I , 2002, IWLCS.

[32]  Ester Bernadó-Mansilla,et al.  Accuracy-Based Learning Classifier Systems: Models, Analysis and Applications to Classification Tasks , 2003, Evolutionary Computation.

[33]  Martin V. Butz,et al.  Analysis and Improvement of Fitness Exploitation in XCS: Bounding Models, Tournament Selection, and Bilateral Accuracy , 2003, Evolutionary Computation.

[34]  Martin V. Butz,et al.  Tournament Selection: Stable Fitness Pressure in XCS , 2003, GECCO.

[35]  Martin V. Butz,et al.  Bounding the Population Size in XCS to Ensure Reproductive Opportunities , 2003, GECCO.

[36]  Stewart W. Wilson,et al.  Advances in Learning Classifier Systems. Fourth International Workshop , 2004 .

[37]  MSc PhD Tim Kovacs BA Strength or Accuracy: Credit Assignment in Learning Classifier Systems , 2004, Distinguished Dissertations.

[38]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.