Using the Taylor expansion of multilayer feedforward neural networks

The Taylor series expansion of continuous functions has sho wn - in many fields - to be an extremely powerful tool to study the characteristics of such functions. This paper ill ustrates the power of the Taylor series expansion of multila yer feedforward neural networks. The paper shows how these expa nsions can be used to investigate positions of decision boundaries, to develop active learning strategies and to pe rform architecture selection.

[1]  Jacek M. Zurada,et al.  Perturbation method for deleting redundant inputs of perceptron networks , 1997, Neurocomputing.

[2]  L. Fu,et al.  Sensitivity analysis for input vector in multilayer feedforward neural networks , 1993, IEEE International Conference on Neural Networks.

[3]  Andries P. Engelbrecht,et al.  Optimizing the number of hidden nodes of a feedforward artificial neural network , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[4]  Patrick Gallinari,et al.  Variable selection with neural networks , 1996, Neurocomputing.

[5]  Yann LeCun,et al.  Improving the convergence of back-propagation learning with second-order methods , 1989 .

[6]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[7]  Ilan Modai,et al.  Sensitivity profile of 41 psychiatric parameters determined by neural network in relation to 8-week outcome , 1995 .

[8]  Andries Petrus Engelbrecht,et al.  Incremental learning using sensitivity analysis , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[9]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[10]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[11]  Andries P. Engelbrecht,et al.  Selective learning using sensitivity analysis , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[12]  Roberto Battiti,et al.  First- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method , 1992, Neural Computation.

[13]  Tiong-Hwee Goh.  Semantic extraction using neural network modelling and sensitivity analysis , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[14]  Frank Stenger,et al.  Cardinal-Type Approximations of a Function and Its Derivatives , 1979 .

[15]  Hans Henrik Thodberg,et al.  Improving Generalization of Neural Networks Through Pruning , 1991, Int. J. Neural Syst..

[16]  Chong-Ho Choi,et al.  Sensitivity analysis of multilayer perceptron with differentiable activation functions , 1992, IEEE Trans. Neural Networks.

[17]  Ferdinand Hergert,et al.  Improving model selection by nonconvergent methods , 1993, Neural Networks.

[18]  R. E. Uhrig,et al.  Sensitivity analysis and applications to nuclear power plant , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[19]  Shigeo Abe,et al.  Input Layer Optimization of Neural Networks by Sensitivity Analysis and its Application to Recognition of Numerals , 1991 .

[20]  Andries P. Engelbrecht,et al.  A sensitivity analysis algorithm for pruning feedforward neural networks , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[21]  Andries Petrus Engelbrecht,et al.  Rule Improvement Through Decision Boundary Detection Using Sensitivity Analysis , 1999, IWANN.

[22]  Andries Petrus Engelbrecht,et al.  Determining the Significance of Input Parameters using Sensitivity Analysis , 1995, IWANN.

[23]  Thomas Czernichow.  Architecture Selection through Statistical Sensitivity Analysis , 1996, ICANN.

[24]  J. N. Kok,et al.  A multilevel nonlinearity study design , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[25]  Gregory J. Wolff,et al.  Optimal Brain Surgeon: Extensions and performance comparisons , 1993, NIPS 1993.

[26]  Alan F. Murray,et al.  International Joint Conference on Neural Networks , 1993 .

[27]  Klaus-Robert Müller,et al.  Asymptotic statistical theory of overtraining and cross-validation , 1997, IEEE Trans. Neural Networks.

[28]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[29]  Ole Winther,et al.  A Quantitative Study Of Pruning By Optimal Brain Damage , 1993, Int. J. Neural Syst..

[30]  Andries Petrus Engelbrecht,et al.  Approximation of a function and its derivatives in feedforward neural networks , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).