Extracted global structure makes local building block processing effective in XCS

Michigan-style learning classifier systems (LCSs), such as the accuracy-based XCS system, evolve distributed problem solutions represented by a population of rules. Recently, it was shown that decomposable problems may require effective processing of subsets of problem attributes, which cannot be generally assured with standard crossover operators. A number of competent crossover operators capable of effective identification and processing of arbitrary subsets of variables or string positions were proposed for genetic and evolutionary algorithms. This paper effectively introduces two competent crossover operators to XCS by incorporating techniques from competent genetic algorithms (GAs): the extended compact GA (ECGA) and the Bayesian optimization algorithm (BOA). Instead of applying standard crossover operators, here a probabilistic model of the global population is built and sampled to generate offspring classifiers locally. Various offspring generation methods are introduced and evaluated. Results indicate that the performance of the proposed learning classifier systems XCS/ECGA and XCS/BOA is similar to that of XCS with informed crossover operators that is given all information about problem structure on input and exploits this knowledge using problem-specific crossover operators.

[1]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[2]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[3]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[4]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[5]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[6]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[7]  Donald A. Waterman,et al.  Pattern-Directed Inference Systems , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[9]  Ronald A. Howard,et al.  Readings on the Principles and Applications of Decision Analysis , 1989 .

[10]  E. Reed The Ecological Approach to Visual Perception , 1989 .

[11]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[12]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[13]  Wray L. Buntine Theory Refinement on Bayesian Networks , 1991, UAI.

[14]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[15]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[16]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[17]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[18]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[19]  Jordan B. Pollack,et al.  Modeling Building-Block Interdependency , 1998, PPSN.

[20]  Stewart W. Wilson Generalization in the XCS Classifier System , 1998 .

[21]  Pier Luca Lanzi,et al.  An Analysis of Generalization in the XCS Classifier System , 1999, Evolutionary Computation.

[22]  Fernando G. Lobo,et al.  Extended Compact Genetic Algorithm in C , 1999 .

[23]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[24]  T. Kovacs Deletion schemes for classifier systems , 1999 .

[25]  Stewart W. Wilson Get Real! XCS with Continuous-Valued Inputs , 1999, Learning Classifier Systems.

[26]  G. Harik Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[27]  Pedro Larrañaga,et al.  Optimization in Continuous Domains by Learning and Simulation of Gaussian Networks , 2000 .

[28]  Stewart W. Wilson Mining Oblique Data with XCS , 2000, IWLCS.

[29]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[30]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[31]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[32]  David E. Goldberg,et al.  Bayesian Optimization Algorithm: From Single Level to Hierarchy , 2002 .

[33]  Pedro Larrañaga,et al.  A Review on Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[34]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[35]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[36]  David E. Goldberg,et al.  Getting the best of both worlds: Discrete and continuous genetic and evolutionary algorithms in concert , 2003, Inf. Sci..

[37]  Martin V. Butz,et al.  Analysis and Improvement of Fitness Exploitation in XCS: Bounding Models, Tournament Selection, and Bilateral Accuracy , 2003, Evolutionary Computation.

[38]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[39]  Martin V. Butz,et al.  Tournament Selection: Stable Fitness Pressure in XCS , 2003, GECCO.

[40]  Ivo Everts,et al.  Extended Compact Genetic Algorithm , 2004 .

[41]  David E. Goldberg,et al.  Real-Coded Bayesian Optimization Algorithm: Bringing the Strength of BOA into the Continuous World , 2004, GECCO.

[42]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[43]  Martin V. Butz,et al.  Toward a theory of generalization and learning in XCS , 2004, IEEE Transactions on Evolutionary Computation.

[44]  Martin V. Butz,et al.  Rule-Based Evolutionary Online Learning Systems: Learning Bounds, Classification, and Prediction , 2004 .

[45]  D. Goldberg,et al.  Bounding Learning Time in XCS , 2004, GECCO.

[46]  Martin Pelikan,et al.  Bayesian Optimization Algorithm , 2005 .

[47]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..