On the LASSO and its Dual

Abstract Proposed by Tibshirani, the least absolute shrinkage and selection operator (LASSO) estimates a vector of regression coefficients by minimizing the residual sum of squares subject to a constraint on the l 1-norm of the coefficient vector. The LASSO estimator typically has one or more zero elements and thus shares characteristics of both shrinkage estimation and variable selection. In this article we treat the LASSO as a convex programming problem and derive its dual. Consideration of the primal and dual problems together leads to important new insights into the characteristics of the LASSO estimator and to an improved method for estimating its covariance matrix. Using these results we also develop an efficient algorithm for computing LASSO estimates which is usable even in cases where the number of regressors exceeds the number of observations. An S-Plus library based on this algorithm is available from StatLib.

[1]  Calyampudi Radhakrishna Rao,et al.  Linear Statistical Inference and its Applications , 1967 .

[2]  C. R. Rao,et al.  Linear Statistical Inference and its Applications , 1968 .

[3]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[4]  M. R. Osborne.  Finite Algorithms in Optimization and Data Analysis , 1985 .

[5]  Simon French,et al.  Finite Algorithms in Optimization and Data Analysis , 1986 .

[6]  A. Gallant,et al.  Nonlinear Statistical Models , 1988 .

[7]  S. Nash,et al.  Linear and Nonlinear Programming , 1987 .

[8]  M. R. Osborne,et al.  On Linear Restricted and Interval Least-Squares Problems , 1988 .

[9]  T. Stamey,et al.  Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. II. Radical prostatectomy treated patients. , 1989, The Journal of urology.

[10]  A. Rademaker,et al.  Re: Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. , 1990, The Journal of urology.

[11]  Alan J. Miller,et al.  Subset Selection in Regression , 1991 .

[12]  Alan J. Miller.  Subset Selection in Regression , 1992 .

[13]  T. Hastie,et al.  [A Statistical View of Some Chemometrics Regression Tools]: Discussion , 1993 .

[14]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[15]  Manfred Deistler,et al.  Statistical modelling and latent variables , 1993 .

[16]  R. Carroll.  Measurement, Regression, and Calibration , 1994 .

[17]  L. Gleser.  Measurement, Regression, and Calibration , 1996 .

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Inge Koch,et al.  On the asymptotic performance of median smoothers in image analysis and nonparametric regression , 1996 .

[20]  Wenjiang J. Fu.  Penalized Regressions: The Bridge versus the Lasso , 1998 .

[21]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[22]  Mary C. Meyer.  An extension of the mixed primal–dual bases algorithm to the case of more constraints than dimensions , 1999 .

[23]  Paul Tseng,et al.  Block coordinate relaxation methods for nonparametric signal denoising with wavelet dictionaries , 2000 .

[24]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[25]  P. Tseng,et al.  Block Coordinate Relaxation Methods for Nonparametric Wavelet Denoising , 2000 .