Probability Matching, the Magnitude of Reinforcement, and Classifier System Bidding

This paper juxtaposes the probability matching paradox of decision theory and the magnitude of reinforcement problem of animal learning theory to show that simple classifier system bidding structures are unable to match the range of behaviors required in the deterministic and probabilistic problems faced by real cognitive systems. The inclusion of a variance-sensitive bidding (VSB) mechanism is suggested, analyzed, and simulated to enable good bidding performance over a wide range of nonstationary probabilistic and deterministic environments.

[1]  J. Goodnow.  Determinants of choice-distribution in two-choice situations. , 1955, The American journal of psychology.

[2]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[3]  H. Simon,et al.  A comparison of game theory and learning theory , 1956 .

[4]  John H. Holland,et al.  Genetic Algorithms and the Optimal Allocation of Trials , 1973, SIAM J. Comput..

[5]  Stewart W. Wilson.  Classifier Systems and the Animat Problem , 1987, Machine Learning.

[6]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[7]  Donald A. Waterman,et al.  Pattern-Directed Inference Systems , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Sidney Siegel,et al.  Theoretical models of choice and strategy behavior: Stable state behavior in the two-choice uncertain outcome situation , 1959 .

[9]  Wayne Lee,et al.  Decision theory and human behavior , 1971 .

[10]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[11]  N. Mackintosh.  The psychology of animal learning , 1974 .

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[14]  James F. Voss,et al.  Effects of instructions in probability learning , 1962 .