Transfer of evolved pattern-based heuristics in games

Learning is key to achieving human-level intelligence. Transferring knowledge that is learned on one task to another one speeds up learning in the target task by exploiting the relevant prior knowledge. As a test case, this study introduces a method to transfer local pattern-based heuristics from a simple board game to a more complex one. The patterns are generated by compositional pattern producing networks (CPPNs), which are evolved with the NEAT neuro-evolution method. Results show that transfer improves both final performance and the total learning time, compared to evolving patterns for the target game from scratch. Pattern-based transfer is therefore a promising approach to scaling up game players toward human-level.

[1]  Shimon Whiteson,et al.  Transfer via inter-task mappings in policy search reinforcement learning , 2007, AAMAS '07.

[2]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[3]  Kenneth O. Stanley and Bobby D. Bryant and Risto Miikkulainen,et al.  Evolving Neural Network Agents in the NERO Video Game , 2005 .

[4]  D. George,et al.  A hierarchical Bayesian model of invariant pattern recognition in the visual cortex , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[5]  Sebastian Thrun,et al.  Explanation-based neural network learning a lifelong learning approach , 1995 .

[6]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Risto Miikkulainen,et al.  Evolving Reusable Neural Modules , 2004, GECCO.

[8]  Peter Stone,et al.  Graph-Based Domain Mapping for Transfer Learning in General Games , 2007, ECML.

[9]  Inman Harvey,et al.  Seeing the light: artificial evolution, real vision , 1994 .

[10]  Peter Stone,et al.  Transfer Learning and Intelligence: an Argument and Approach , 2008, AGI.

[11]  Kenneth O. Stanley,et al.  A novel generative encoding for exploiting neural network sensor and output geometry , 2007, GECCO '07.

[12]  DeLiang Wang,et al.  Incremental learning of complex temporal patterns , 1996, IEEE Trans. Neural Networks.

[13]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[14]  Kenneth O. Stanley,et al.  Compositional Pattern Producing Networks : A Novel Abstraction of Development , 2007 .

[15]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[16]  Marco Dorigo,et al.  Incremental Evolution of Robot Controllers for a Highly Integrated Task , 2006, SAB.

[17]  Susan L. Epstein,et al.  Learning Game-Specific Spatially-Oriented Heuristics , 1998, Constraints.

[18]  Michael R. Genesereth,et al.  General Game Playing: Overview of the AAAI Competition , 2005, AI Mag..

[19]  Risto Miikkulainen,et al.  Evolving a real-world vehicle warning system , 2006, GECCO.

[20]  Susan L. Epstein.  For the Right Reasons: The FORR Architecture for Learning in a Skill Domain , 1994, Cogn. Sci..

[21]  Kenneth O. Stanley,et al.  A Case Study on the Critical Role of Geometric Regularity in Machine Learning , 2008, AAAI.

[22]  Risto Miikkulainen,et al.  Coevolving Strategies for General Game Playing , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[23]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[24]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..