Classifier prediction based on tile coding

This paper introduces XCSF extended with tile coding prediction: each classifier implements a tile coding approximator; the genetic algorithm is used to adapt both classifier conditions (i.e., to partition the problem) and the parameters of each approximator; thus XCSF evolves an ensemble of tile coding approximators instead of the typical monolithic approximator used in reinforcement learning. The paper reports a comparison between (i) XCSF with tile coding prediction and (ii) plain tile coding. The results show that XCSF with tile coding always reaches optimal performance, it usually learns as fast as the best parametrized tile coding, and it can be faster than the typical tile coding setting. In addition, the analysis of the evolved tile coding ensembles shows that XCSF actually adapts local approximators following what is currently considered the best strategy to adapt the tile coding parameters in a given problem.

[1]  Andrew W. Moore,et al.  Generalization in Reinforcement Learning: Safely Approximating the Value Function , 1994, NIPS.

[2]  Peter Stone,et al.  Function Approximation via Tile Coding: Automating Parameter Choice , 2005, SARA.

[3]  Richard S. Sutton,et al.  Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding , 1995, NIPS.

[4]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[5]  Daniele Loiacono,et al.  Extending XCSF beyond linear approximation , 2005, GECCO '05.

[6]  Stewart W. Wilson.  Classifiers that approximate functions , 2002, Natural Computing.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Daniele Loiacono,et al.  XCS with computed prediction in multistep environments , 2005, GECCO '05.

[9]  Stuart I. Reynolds.  Reinforcement Learning with Exploration , 2002 .

[10]  Martin V. Butz,et al.  An algorithmic description of XCS , 2000, Soft Comput..

[11]  Xiaoqin Zhang,et al.  A Randomized ANOVA Procedure for Comparing Performance Curves , 1997, ICML.

[12]  S. Glantz.  Primer of applied regression and analysis of variance / Stanton A. Glantz, Bryan K. Slinker , 1990 .

[13]  Lashon B. Booker.  Approximating Value Functions in Classifier Systems , 2005 .

[14]  Daniele Loiacono,et al.  XCS with computed prediction for the learning of Boolean functions , 2005, 2005 IEEE Congress on Evolutionary Computation.

[15]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .