THE LASSO AND RELATED METHODS 2 1 The lasso and related methods

We consider the least angle regression and forward stagewise algorithms for solving penalized least squares regression problems. In Efron et al. (2004) it is proven that the least angle regression algorithm, with a small modification, solves the lasso (L1 constrained) regression problem. Here we give an analogous result for incremental forward stagewise regression, showing that it fits a monotone version of the lasso. We also study a condition under which the coefficient paths of the lasso are monotone, and hence the different algorithms all coincide. Finally, we compare the lasso and forward stagewise procedures in a simulation study involving a large number of correlated predictors.