Evolution and design of distributed learning rules

The paper describes the application of neural networks as learning rules for the training of neural networks. The learning rule is part of the neural network architecture. As a result the learning rule is non-local and globally distributed within the network. The learning rules are evolved using an evolution strategy. The survival of a learning rule is based on its performance in training neural networks on a set of tasks. Training algorithms will be evolved for single layer artificial neural networks. Experimental results show that a learning rule of this type is very capable of generating an efficient training algorithm.

[1]  Bernard Widrow,et al.  Punish/Reward: Learning with a Critic in Adaptive Threshold Systems , 1973, IEEE Trans. Syst. Man Cybern..

[2]  P. Anandan,et al.  Pattern-recognizing stochastic learning automata , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[4]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[5]  Kurt Hornik,et al.  FEED FORWARD NETWORKS ARE UNIVERSAL APPROXIMATORS , 1989 .

[6]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[7]  Michael I. Jordan,et al.  A more biologically plausible learning rule for neural networks. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[8]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[9]  Jonathan Baxter.  The evolution of learning algorithms for artificial neural networks , 1993 .

[10]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[11]  Kishan G. Mehrotra,et al.  Elements of artificial neural networks , 1996 .

[12]  Thomas Bäck,et al.  Evolutionary Algorithms in Theory and Practice , 1996 .

[13]  D. P. Gallogly,et al.  Computational analyses in cognitive neuroscience: In defense of biological implausibility , 1999, Psychonomic bulletin & review.

[14]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.