Reinforcement learning with classifier systems: Adaptive default hierarchy formation

Autonomous systems are likely to be required to face situations that cannot be foreseen by their designers. The potential for perpetually novel situations places a premium on mechanisms that allow for automatic adaptation in a general setting. The term reinforcement learning problems (Mendel and McLaren, 1970) generally describes problems where a control system must adapt based on performance-only feedback. This paper considers the learning classifier system (LCS) as an approach to reinforcement learning problems. An LCS is a type of adaptive expert system that uses a knowledge base of production rules in a low-level syntax that can be manipulated by a genetic algorithm (GA) (Holland. 1975; Goldberg, 1989) Genetic algorithms comprise a class of computerized search procedures that are based on the mechanics of natural genetics (Goldberg, 1989; Holland. 1975). An important feature of the LCS paradigm is the possible adaptive formation of default hierarchies (layered sets of default and exception rules) )Hol...

[1]  Lawrence Davis,et al.  Mapping Neural Networks into Classifier Systems , 1989, ICGA.

[2]  Lawrence Davis,et al.  Mapping Classifier Systems Into Neural Networks , 1988, NIPS.

[3]  John H. Holland,et al.  Empirical studies of default hierarchies and sequences of rules in learning classifier systems , 1988 .

[4]  Stewart W. Wilson Knowledge Growth in an Artificial Animal , 1985, ICGA.

[5]  Richard S. Sutton,et al.  Learning and Sequential Decision Making , 1989 .

[6]  Stewart W. Wilson Hierarchical Credit Allocation in a Classifier System , 1987, IJCAI.

[7]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[8]  Emil L. Post Formal Reductions of the General Combinatorial Decision Problem , 1943 .

[9]  Andrew G. Barto,et al.  Connectionist learning for control: an overview , 1990 .

[10]  Jerry M. Mendel,et al.  Reinforcement-learning control and pattern recognition systems , 1994 .

[11]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[12]  Gunar E. Liepins,et al.  A Classifier Based System for Discovering Scheduling Heuristics , 1987, ICGA.

[13]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[14]  John H. Holland Genetic Algorithms and Classifier Systems: Foundations and Future Directions , 1987, ICGA.

[15]  Kumpati S. Narendra,et al.  Recent Developments in Learning Automata , 1986 .

[16]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .