This paper presents a novel incremental algorithm that combines Q-learning, a well-known dynamic programming-based reinforcement learning method, with the TD() return estimation process, which is typically used in actor-critic learning, another well-known dynamic programming-based reinforcement learning method. The parameter is used to distribute credit throughout sequences of actions, leading to faster learning and also helping to alleviate the non-Markovian eeect of coarse state-space quantization. The resulting algorithm, Q()-learning, thus combines some of the best features of the Q-learning and actor-critic learning paradigms. The behavior of this algorithm has been demonstrated through computer simulations.
[1]
Ben J. A. Kröse,et al.
Learning from delayed rewards
,
1995,
Robotics Auton. Syst..
[2]
Mahesan Niranjan,et al.
On-line Q-learning using connectionist systems
,
1994
.
[3]
Mark D. Pendrith.
On Reinforcement Learning of Control Actions in Noisy and Non-Markovian Domains
,
1994
.
[4]
J. Peng,et al.
Efficient Learning and Planning Within the Dyna Framework
,
1993,
IEEE International Conference on Neural Networks.
[5]
Long-Ji Lin,et al.
Reinforcement learning for robots using neural networks
,
1992
.
[6]
Richard S. Sutton,et al.
Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming
,
1990,
ML.