论文信息 - Some numerical aspects of the training problem for feed-forward neural nets

Some numerical aspects of the training problem for feed-forward neural nets

This paper considers the feed-forward training problem from the numerical point of view, in particular the conditioning of the problem. It is well known that the feed-forward training problem is often ill-conditioned; this affects the behaviour of training algorithms, the choice of such algorithms and the quality of the solutions achieved. A geometric interpretation of ill-conditioning is explored and an example of function approximation is analysed in detail.

[1] D. J. Mills,et al. Neural networks and nonlinear optimization i the representation of continuous functions , 1992 .

[2] J. G. Taylor,et al. Mathematical Approaches to Neural Networks , 1993 .

[3] G. T. Timmer,et al. Stochastic global optimization methods part I: Clustering methods , 1987, Math. Program..

[4] S. W. Ellacott,et al. The numerical analysis approach , 1993 .

[5] Adrian J. Shepherd,et al. A CLASSICAL ALGORITHM FOR AVOIDING LOCAL MINIMA , 1994 .

[6] Patrick van der Smagt. Minimisation methods for training feedforward neural networks , 1994, Neural Networks.

[7] Alan J. Mayne,et al. Towards Global Optimisation 2 , 1976 .

[8] Jon G. Rokne,et al. New computer methods for global optimization , 1988 .

[9] Aimo A. Törn,et al. Global Optimization , 1999, Science.

[10] J. J. McKeown,et al. Specialised versus general-purpose algorithms for minimising functions that are sums of squared terms , 1975, Math. Program..

[11] George Cybenko,et al. Ill-Conditioning in Neural Network Training Problems , 1993, SIAM J. Sci. Comput..

[12] D. Sprevak,et al. An introduction to unconstrained optimisation: a computer illustrated text , by J. McKeown, D. Meegan and D. Sprevak. Pp 122 + disc. £14·50, £25·50. 1990. ISBN 0-7503-0025-6, -0026-4 (Adam Hilger) , 1991, The Mathematical Gazette.