Knowledge Extraction and Problem Structure Identification in XCS

XCS has been shown to solve hard problems in a machine-learning competitive way. Recent theoretical advancements show that the system can scale-up polynomially in the problem complexity and problem size given the problem is a k-DNF with certain properties. This paper addresses two major issues in XCS: (1) knowledge extraction and (2) structure identification. Knowledge extraction addresses the issue of mining problem knowledge from the final solution developed by XCS. The goal is to identify most important features in the problem and the dependencies among those features. The extracted knowledge may not only be used for further data mining, but may actually be re-fed into the system giving it further competence in solving problems in which dependent features, that is, building blocks, need to be processed effectively. This paper proposes to extract a feature dependency tree out of the developed rule-based problem representation of XCS. The investigations herein focus on Boolean function problems. The extension to nominal and real-valued features is discussed.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[3]  Stewart W. Wilson.  Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[4]  David E. Goldberg,et al.  The Race, the Hurdle, and the Sweet Spot , 1998 .

[5]  Stewart W. Wilson.  Generalization in the XCS Classifier System , 1998 .

[6]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[7]  Peter J. Bentley,et al.  Evolutionary Design By Computers , 1999 .

[8]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[9]  Xavier Llorà,et al.  XCS and GALE: A Comparative Study of Two Learning Classifier Systems on Data Mining , 2001, IWLCS.

[10]  Martin J. Oates,et al.  A Preliminary Investigation of Modified XCS as a Generic Data Mining Tool , 2001, IWLCS.

[11]  Stewart W. Wilson.  Compact Rulesets from XCSI , 2001, IWLCS.

[12]  Martin J. Oates,et al.  A Ruleset Reduction Algorithm for the XCS Learning Classifier System , 2002, IWLCS.

[13]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[14]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[15]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[16]  Ester Bernadó-Mansilla,et al.  Accuracy-Based Learning Classifier Systems: Models, Analysis and Applications to Classification Tasks , 2003, Evolutionary Computation.

[17]  Martin V. Butz,et al.  Analysis and Improvement of Fitness Exploitation in XCS: Bounding Models, Tournament Selection, and Bilateral Accuracy , 2003, Evolutionary Computation.

[18]  Xavier Llorà,et al.  Wise Breeding GA via Machine Learning Techniques for Function Optimization , 2003, GECCO.

[19]  Martin V. Butz,et al.  Toward a theory of generalization and learning in XCS , 2004, IEEE Transactions on Evolutionary Computation.

[20]  Stewart W. Wilson,et al.  Advances in Learning Classifier Systems. Fourth International Workshop , 2004 .

[21]  D. Goldberg,et al.  Bounding Learning Time in XCS , 2004, GECCO.