Projection pursuit learning networks for regression

Two types of learning networks for nonparametric regression problems are studied and compared: one is the parametric two-layer perceptron type neural network, which is well known in artificial neural network (ANN) literature; the other is the semiparametric projection pursuit network (PPN), which has emerged in recent years in the statistical estimation literature. From an algorithmic viewpoint, both the PPN and the ANN parametrically form projections of the data in directions determined from interconnection weights. However, unlike an ANN which uses a fixed set of nonlinear nodal functions to perform an explicit parametric estimate of a nonparametric model, the PPN nonparametrically estimates the nonlinear functions using a one-dimensional data smoother. From experimental simulations, ANNs and PPNs perform comparably in predicting independent test data but PPN training is much faster than that of an ANN.<<ETX>>

[1]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[2]  R. Tibshirani.  Estimating Transformations for Regression via Additivity and Variance Stabilization , 1988 .

[3]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[6]  Tomas Hrycej,et al.  A modular architecture for efficient learning , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[7]  Jerome H. Friedman,et al.  Smoothing of Scatterplots , 1982 .

[8]  Etienne Barnard,et al.  Optimization for training neural nets , 1992, IEEE Trans. Neural Networks.

[9]  J. Friedman.  A VARIABLE SPAN SMOOTHER , 1984 .

[10]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[11]  I. Johnstone,et al.  Projection-Based Approximation and a Duality with Kernel Methods , 1989 .

[12]  Jenq-Neng Hwang,et al.  A unifying viewpoint of multilayer perceptrons and hidden Markov models , 1989, IEEE International Symposium on Circuits and Systems,.

[13]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[14]  J. Friedman,et al.  Estimating Optimal Transformations for Multiple Regression and Correlation. , 1985 .

[15]  Allan R. Wilks,et al.  The new S language: a programming environment for data analysis and graphics , 1988 .

[16]  L. Jones.  On a conjecture of Huber concerning the convergence of projection pursuit regression , 1987 .

[17]  Raymond L. Watrous.  Learning Algorithms for Connectionist Networks: Applied Gradient Methods of Nonlinear Optimization , 1988 .

[18]  P. S. Lewis,et al.  Function approximation and time series prediction with neural networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[19]  Hecht-Nielsen.  Theory of the backpropagation neural network , 1989 .

[20]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[21]  R. Hecht-Nielsen,et al.  Theory of the Back Propagation Neural Network , 1989 .

[22]  Stefanos Kollias,et al.  An adaptive least squares algorithm for the efficient training of artificial neural networks , 1989 .