Fast Reinforcement Learning through Eugenic Neuro-Evolution

In this paper we introduce EuSANE, a novel reinforcement learning algorithm based on the SANE neuro-evolution method. It uses a global search algorithm, the Eugenic Algorithm, to optimize the selection of neurons to the hidden layer of SANE networks. The performance of EuSANE is evaluated in the two-pole balancing benchmark task, showing that EuSANE is signiicantly stronger than other reinforcement learning methods to date in this task.

[1]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[2]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[3]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[4]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[5]  Alexis P. Wieland,et al.  Evolving Controls for Unstable Systems , 1991 .

[6]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[7]  Gilbert Syswerda,et al.  Simulated Crossover in Genetic Algorithms , 1992, FOGA.

[8]  Daniel Polani,et al.  Adaptation of Kohonen Feature Map Topologies by Genetic Algorithms , 1992, PPSN.

[9]  Daniel Polani,et al.  Training Kohonen Feature Maps in Different Topologies: An Analysis Using Genetic Algorithms , 1993, ICGA.

[10]  Mark D. Pendrith.  On Reinforcement Learning of Control Actions in Noisy and Non-Markovian Domains , 1994 .

[11]  L. Darrell Whitley,et al.  Cellular Encoding Applied to Neurocontrol , 1995, ICGA.

[12]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[13]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 1996, Machine Learning.

[14]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[15]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[16]  David E. Moriarty,et al.  Symbiotic Evolution of Neural Networks in Sequential Decision Tasks , 1997 .

[17]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[18]  John W Prior.  Eugenic Evolution for Combinatorial Optimization , 1998 .

[19]  L. D. Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.