论文信息 - Iterative Construction of Sparse Polynomial Approximations

Iterative Construction of Sparse Polynomial Approximations

We present an iterative algorithm for nonlinear regression based on construction of sparse polynomials. Polynomials are built sequentially from lower to higher order. Selection of new terms is accomplished using a novel look-ahead approach that predicts whether a variable contributes to the remaining error. The algorithm is based on the tree-growing heuristic in LMS Trees which we have extended to approximation of arbitrary polynomials of the input features. In addition, we provide a new theoretical justification for this heuristic approach. The algorithm is shown to discover a known polynomial from samples, and to make accurate estimates of pixel values in an image-processing task.

[1] Dennis Gabor,et al. A universal nonlinear filter, predictor and simulator which optimizes itself by a learning process , 1961 .

[2] A. G. Ivakhnenko,et al. Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[3] Saburo Ikeda,et al. Sequential GMDH Algorithm and Its Application to River Flow Prediction , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[4] Terence D. Sanger,et al. Basis-Function Trees as a Generalization of Local Variable Selection Methods , 1990, NIPS.

[5] Marek Karpinski,et al. Fast Parallel Algorithms for Sparse Multivariate Polynomial Interpolation over Finite Fields , 1988, SIAM J. Comput..

[6] Richard S. Sutton,et al. Learning Polynomial Functions by Feature Construction , 1991, ML.

[7] Terence D. Sanger,et al. A tree-structured adaptive network for function approximation in high-dimensional spaces , 1991, IEEE Trans. Neural Networks.