Automatic smoothing parameter selection: A survey

This is a survey of recent developments in smoothing parameter selection for curve estimation. The first goal of this paper is to provide an introduction to the methods available, with discussion at both a practical and also a nontechnical level, including comparison of methods. The second goal is to provide access to the literature, especially on smoothing parameter selection, but also on curve estimation in general. The two main settings considered here are nonparametric regression and probability density estimation, although the points made apply to other settings as well. These points also apply to many different estimators, although the focus is on kernel estimators, because they are the most easily understood and motivated, and have been at the heart of the development in the field.

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[3]  E. Nadaraya.  On Estimating Regression , 1964 .

[4]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[5]  Gerald S. Rogers,et al.  Mathematical Statistics: A Decision Theoretic Approach , 1967 .

[6]  M. Stone,et al.  Mathematical Statistics: A Decision Theoretic Approach , 1968 .

[7]  M. Woodroofe.  On Choosing a Delta-Sequence , 1970 .

[8]  E. Wegman.  Nonparametric Probability Density Estimation: I. A Summary of Available Methods , 1972 .

[9]  C. L. Mallows.  Some comments on C_p , 1973 .

[10]  C. L. Mallows.  Some Comments onCp , 1973 .

[11]  Leopold Schmetterer,et al.  COMPSTAT 1974 : proceedings in computational statistics , 1974 .

[12]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[13]  R. M. Clark.  A calibration curve for radiocarbon dates , 1975, Antiquity.

[14]  G. Wahba,et al.  A completely automatic french curve: fitting spline functions by cross validation , 1975 .

[15]  R. Kronmal,et al.  An Introduction to the Implementation and Theory of Nonparametric Density Estimation , 1976 .

[16]  Robert P. W. Duin,et al.  On the Choice of Smoothing Parameters for Parzen Estimators of Probability Density Functions , 1976, IEEE Transactions on Computers.

[17]  D. W. Scott,et al.  Kernel density estimation revisited , 1977 .

[18]  M. J. Fryer.  A Review of Some Non-parametric Methods of Density Estimation , 1977 .

[19]  H. D. Brunk.  Univariate density estimation by orthogonal series , 1978 .

[20]  W. Wertz.  Statistical density estimation: A survey , 1978 .

[21]  R. Tapia,et al.  Nonparametric Probability Density Estimation , 1978 .

[22]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[23]  H. Müller,et al.  Kernel estimation of regression functions , 1979 .

[24]  C. Tsokos,et al.  Developments in Nonparametric Density-Estimation , 1980 .

[25]  William F. Eddy.  Computer Science and Statistics: Proceedings of the 13th Symposium on the Interface , 1981 .

[26]  E. F. Schuster,et al.  On the Nonconsistency of Maximum Likelihood Nonparametric Density Estimators , 1981 .

[27]  G. Collomb.  Estimation Non-paramétrique de la Régression: Revue Bibliographique@@@Estimation Non-parametrique de la Regression: Revue Bibliographique , 1981 .

[28]  David W. Scott,et al.  Monte Carlo Study of Three Data-Based Nonparametric Probability Density Estimators , 1981 .

[29]  A. Krieger,et al.  Weak Convergence and Efficient Density Estimation at a Point , 1981 .

[30]  M. Rudemo.  Empirical Choice of Histograms and Kernel Density Estimators , 1982 .

[31]  Peter Hall,et al.  Cross-validation in density estimation , 1982 .

[32]  S. Sheather.  A data-based algorithm for choosing the window width when estimating the density at a point , 1983 .

[33]  P. Hall.  Large Sample Optimality of Least Squares Cross-Validation in Density Estimation , 1983 .

[34]  Prakasa Rao.  Nonparametric functional estimation , 1983 .

[35]  S. Geman,et al.  Consistent Cross-Validated Density Estimation , 1983 .

[36]  Ker-Chau Li,et al.  The Data-Smoothing Aspect of Stein Estimates , 1984 .

[37]  J. Anderson.  Regression and Ordered Categorical Variables , 1984 .

[38]  J. Rice.  Bandwidth Choice for Nonparametric Regression , 1984 .

[39]  A. Bowman.  An alternative method of cross-validation for the smoothing of density estimates , 1984 .

[40]  C. J. Stone,et al.  An Asymptotically Optimal Window Selection Rule for Kernel Density Estimates , 1984 .

[41]  Ker-Chau Li,et al.  From Stein's Unbiased Risk Estimates to the Method of Generalized Cross Validation , 1985 .

[42]  Gérard Collomb,et al.  Nonparametric regression: An up–to–date bibliography , 1985 .

[43]  D. W. Scott,et al.  Oversmoothed Nonparametric Density Estimates , 1985 .

[44]  P. Burman.  A data dependent approach to density estimation , 1985 .

[45]  Luc Devroye,et al.  Nonparametric Density Estimation , 1985 .

[46]  W. Härdle,et al.  Asymptotic nonequivalence of some bandwidth selectors in nonparametric regression , 1985 .

[47]  J. Marron.  An Asymptotically Efficient Solution to the Bandwidth Problem of Kernel Density Estimation , 1985 .

[48]  W. Härdle,et al.  Optimal Bandwidth Selection in Nonparametric Regression Function Estimation , 1985 .

[49]  J. Rice.  Bandwidth choice for differentiation , 1986 .

[50]  T. Gasser,et al.  Residual variance and residual pattern in nonlinear regression , 1986 .

[51]  J. Stephen Marron.  What does optimal bandwidth selection mean for nonparametric regression estimation , 1986 .

[52]  James Stephen Marron,et al.  A Comparison of Cross-Validation Techniques in Density Estimation , 1987 .

[53]  A nonparametric data based univariate density function estimate , 1987 .

[54]  J. Marron.  Partitioned cross-validation , 1987 .

[55]  P. K. Bhattacharya,et al.  Weak Convergence of $k$-NN Density and Regression Estimators with Varying $k$ and Applications , 1987 .

[56]  J. Marron,et al.  Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation , 1987 .

[57]  H. Müller,et al.  Variable Bandwidth Kernel Estimators of Regression Curves , 1987 .

[58]  James Stephen Marron,et al.  On the Amount of Noise Inherent in Bandwidth Selection for a Kernel Density Estimator , 1987 .

[59]  L. Devroye.  A Course in Density Estimation , 1987 .

[60]  James Stephen Marron,et al.  Asymptotically Optimal Bandwidth Selection for Kernel Density Estimators from Randomly Right-Censored Samples. , 1987 .

[61]  On the use of compactly supported density estimates in problems of discrimination , 1987 .

[62]  D. W. Scott,et al.  Biased and Unbiased Cross-Validation in Density Estimation , 1987 .

[63]  James Stephen Marron,et al.  Estimation of integrated squared density derivatives , 1987 .

[64]  H. Bierens.  Advances in Econometrics: Kernel estimators of regression functions , 1987 .

[65]  D. Pollard,et al.  $U$-Processes: Rates of Convergence , 1987 .

[66]  Y. Dodge.  on Statistical data analysis based on the L1-norm and related methods , 1987 .

[67]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[68]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .

[69]  Ulrich Stadtmüller,et al.  Bandwidth choice and confidence intervals for derivatives of noisy data , 1987 .

[70]  P. Hall.  On Kullback-Leibler loss and density estimation , 1987 .

[71]  W. Härdle,et al.  How Far are Automatically Chosen Regression Smoothing Parameters from their Optimum , 1988 .

[72]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[73]  Matt P. Wand,et al.  Minimizing L 1 distance in nonparametric density estimation , 1988 .

[74]  David W. Scott.  How Far Are Automatically Chosen Regression Smoothing Parameters From Their Optimum?: Comment , 1988 .

[75]  Matt P. Wand,et al.  On the minimization of absolute distance in kernel density estimation , 1988 .

[76]  A. Ullah.  Non-parametric estimation of econometric functionals , 1988 .

[77]  On the Choice of the Bandwidth in Kernel Nonparametric Regression , 1988 .

[78]  J. Marron.  Improvement of a data based bandwidth selector , 1988 .

[79]  M. C. Jones,et al.  Spline Smoothing and Nonparametric Regression. , 1989 .

[80]  M. Wand,et al.  Bandwidth choice for density derivatives , 1990 .

[81]  S.,et al.  CONSISTENT CROSS-VALIDATED DENSITY ESTIMATION , 2022 .