The compact classifier system: scalability analysis and first results

This paper presents an analysis of how maximally general and accurate rules can be evolved in a Pittsburgh-style classifier system. In order to be able to perform such an analysis we introduce a simple bare-bones Pittsburgh-style classifier systems - the compact classifier system (CCS) - based on estimation of distribution algorithms. Using a common rule encoding schemes of Pittsburgh-style classifier systems, CCS maintains a dynamic set of probability vectors that compactly describe a rule set. The compact genetic algorithm is used to evolve each of the initially perturbated probability vectors. Results show how CCS is able to evolve in a compact, simple, and elegant manner rule sets composed by maximally general and accurate rules. The initial theoretical analysis and results also show that traditional encoding schemes used by Pittsburgh-style classifiers add an extra facet of difficulty. Such a bias plays a central role on the overall performance and scalability of CCS and other Pittsburgh-style systems using such encoding schemes

[1]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[2]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[3]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[4]  Xavier Llorà,et al.  Knowledge-independent data mining with fine-grained parallel evolutionary algorithms , 2001 .

[5]  Xavier Llorà,et al.  Binary rule encoding schemes: a study using the compact classifier system , 2005, GECCO '05.

[6]  E. Cantu-Paz,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1997, Evolutionary Computation.

[7]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[8]  David E. Goldberg,et al.  The gambler''s ruin problem , 1997 .

[9]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[10]  Luca Lanzi Pier,et al.  Extending the Representation of Classifier Conditions Part II: From Messy Coding to S-Expressions , 1999 .

[11]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[12]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[13]  P. Lanzi Extending the representation of classifier conditions part I: from binary to messy coding , 1999 .

[14]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[15]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[16]  Stewart W. Wilson ZCS: A Zeroth Level Classifier System , 1994, Evolutionary Computation.

[17]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[18]  Stewart W. Wilson Get Real! XCS with Continuous-Valued Inputs , 1999, Learning Classifier Systems.

[19]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[20]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[21]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[22]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[23]  C. Janikow A Knowledge-Intensive Genetic Algorithm for Supervised Learning , 2004, Machine Learning.