Sparse regularized local regression

The intention is to provide a Bayesian formulation of regularized local linear regression, combined with techniques for optimal bandwidth selection. This approach arises from the idea that only those covariates that are found to be relevant for the regression function should be considered by the kernel function used to define the neighborhood of the point of interest. However, the regression function itself depends on the kernel function. A maximum posterior joint estimation of the regression parameters is given. Also, an alternative algorithm based on sampling techniques is developed for finding both the regression parameter distribution and the predictive distribution.

[1]  M. Wand,et al.  Multivariate plug-in bandwidth selection , 1994 .

[2]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[3]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[4]  D. W. Scott,et al.  Cross-Validation of Multivariate Densities , 1994 .

[5]  Hao Helen Zhang,et al.  Component selection and smoothing in smoothing spline analysis of variance models -- COSSO , 2003 .

[6]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[7]  Jeffrey S. Racine,et al.  Nonparametric Estimation of Regression Functions in the Presence of Irrelevant Regressors , 2007, The Review of Economics and Statistics.

[8]  W. Cleveland,et al.  Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting , 1988 .

[9]  Ivana Horová,et al.  Full bandwidth matrix selectors for gradient kernel density estimate , 2013, Comput. Stat. Data Anal..

[10]  T. Hastie,et al.  Local Regression: Automatic Kernel Carpentry , 1993 .

[11]  Lijian Yang,et al.  Multivariate bandwidth selection for local linear regression , 1999 .

[12]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[13]  Jianqing Fan Local Linear Regression Smoothers and Their Minimax Efficiencies , 1993 .

[14]  Simen Gaure,et al.  OLS with multiple high dimensional category variables , 2013, Comput. Stat. Data Anal..

[15]  M. Wand Local Regression and Likelihood , 2001 .

[16]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[17]  M. Wand,et al.  Multivariate Locally Weighted Least Squares Regression , 1994 .

[18]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[19]  Hao Helen Zhang,et al.  Component selection and smoothing in multivariate nonparametric regression , 2006, math/0702659.

[20]  J. Lafferty,et al.  Rodeo: Sparse, greedy nonparametric regression , 2008, 0803.1709.

[21]  William E. Strawderman,et al.  Minimax Adaptive Generalized Ridge Regression Estimators , 1978 .

[22]  Concha Bielza,et al.  Lazy lasso for local regression , 2012, Comput. Stat..