Successful cooperation between heterogeneous fuzzy Q-learning agents

Cooperation in learning improves the speed of convergence and the quality of learning. Special treatment is needed when heterogeneous agents cooperate in learning. It has been discussed that, cooperation in learning may cause the learning process not to converge if heterogeneity is not handled properly. In this paper, it is assumed that two (or several) heterogeneous Q-learning agents cooperate to learn. The two hunter agents independently pursue a prey agent on a two-dimensional lattice: however, the hunters' visual-field depths are different. Thus, in order to have successful cooperation, the agents should be able to interpret other agents' Q-table. For this purpose, an algorithm has been proposed and implemented on the pursuit problem. Two case studies has been introduced and simulated to show the effectiveness of the proposed algorithm.

[1]  Victor R. Lesser,et al.  Sharing Metainformation to Guide Cooperative Search Among Heterogeneous Reusable Agents , 1997, IEEE Trans. Knowl. Data Eng..

[2]  Ming Tan,et al.  Multi-Agent Reinforcement Learning: Independent versus Cooperative Agents , 1997, ICML.

[3]  Hisao Ishibuchi,et al.  Implementation of fuzzy Q-learning for a soccer agent , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[4]  Victor R. Lesser,et al.  Understanding the Role of Negotiation in Distributed Search Among Heterogereous Agents , 1993, IJCAI.

[5]  William J. Frawley,et al.  ILS: A system of learning distributed heterogeneous agents for network traffic management , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[6]  Susan E. Lander,et al.  Distributed search and conflict management among reusable heterogeneous agents , 1995 .

[7]  Majid Nili Ahmadabadi,et al.  Expertness based cooperative Q-learning , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[8]  Seiji Yamada,et al.  Experimental comparison of a heterogeneous learning multi-agent system with a homogeneous one , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).

[9]  Maram V. Nagendraprasad,et al.  Learning situtation-specific control in multi-agent systems , 1997 .