Heterogeneous and Hierarchical Cooperative Learning via Combining Decision Trees

Decision trees, being human readable and hierarchically structured, provide a suitable mean to derive state-space abstraction and simplify the inclusion of the available knowledge for a reinforcement learning (RL) agent. In this paper, we address two approaches to combine and purify the available knowledge in the abstraction trees, stored among different RL agents in a multi-agent system, or among the decision trees learned by the same agent using different methods. Simulation results in nondeterministic football learning task provide strong evidences for enhancement in convergence rate and policy performance

[1]  Nitesh V. Chawla,et al.  Decision tree learning on very large data sets , 1998, SMC.

[2]  Roland Siegwart,et al.  Cooperative Learning for Very Long Learning Tasks: A Society Inspired Approach to Persistence of Knowledge , 2005 .

[3]  Ming Tan,et al.  Multi-Agent Reinforcement Learning: Independent versus Cooperative Agents , 1997, ICML.

[4]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[5]  M. N. Ahmadabadi,et al.  An extension of weighted strategy sharing in cooperative Q-learning for specialized agents , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[6]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[7]  Majid Nili Ahmadabadi,et al.  Cooperative Q-learning: the knowledge sharing issue , 2001, Adv. Robotics.

[8]  Andrew W. Moore,et al.  The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces , 2004, Machine Learning.

[9]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[10]  Andrew McCallum,et al.  Overcoming Incomplete Perception with Utile Distinction Memory , 1993, ICML.

[11]  Jude W. Shavlik,et al.  Creating Advice-Taking Reinforcement Learners , 1998, Machine Learning.

[12]  Majid Nili Ahmadabadi,et al.  Expertness based cooperative Q-learning , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[13]  Manuela M. Veloso,et al.  Tree Based Discretization for Continuous State Space Reinforcement Learning , 1998, AAAI/IAAI.

[14]  Majid Nili Ahmadabadi,et al.  Expertness measuring in cooperative learning , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[15]  Leslie Pack Kaelbling,et al.  Input Generalization in Delayed Reinforcement Learning: An Algorithm and Performance Comparisons , 1991, IJCAI.

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[17]  Gillian M. Hayes,et al.  A Robot Controller Using Learning by Imitation , 1994 .

[18]  Majid Nili Ahmadabadi,et al.  Reduction of Learning Time for Robots Using Automatic State Abstraction , 2006, EUROS.

[19]  Stuart J. Russell,et al.  Reinforcement Learning with Hierarchies of Machines , 1997, NIPS.

[20]  Thomas G. Dietterich Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition , 1999, J. Artif. Intell. Res..

[21]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .