论文信息 - The N-2-N Encoder: A Matter of Representation

The N-2-N Encoder: A Matter of Representation

Kruglyak [4] demonstrated that weights exist to implement the N-2-N encoder for all finite N, hut it is known that Backpropagation (BP) is unusually poor at learning such solutions for N > 8 ([6]). We show that the learning problem lies not with BP, but with the pattern representation typically used in the encoder task. With an appropriate representation, we demonstrate that BP can learn encoders in approximately linear time with N as large as 100. This underlines yet again the crucial importance of pattern representation in neural network learning.

[1] Geoffrey E. Hinton,et al. Learning internal representations by error propagation , 1986 .

[2] Anders Krogh,et al. Introduction to the theory of neural computation , 1994, The advanced book program.

[3] James L. McClelland,et al. Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[4] Geoffrey E. Hinton,et al. Proceedings of the 1988 Connectionist Models Summer School , 1989 .

[5] Raymond Lister,et al. Visualizing weight dynamics in the N-2-N encoder , 1993, IEEE International Conference on Neural Networks.

[6] S. Fahlman. Fast-learning variations on back propagation: an empirical study. , 1989 .

[7] Geoffrey E. Hinton,et al. A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[8] Leonid Kruglyak,et al. How to Solve the N Bit Encoder Problem with Just Two Hidden Units , 1990, Neural Computation.

[9] James L. McClelland,et al. James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.