An Ordered Lasso and Sparse Time-Lagged Regression

We consider regression scenarios where it is natural to impose an order constraint on the coefficients. We propose an order-constrained version of ℓ1-regularized regression (Lasso) for this problem, and show how to solve it efficiently using the well-known pool adjacent violators algorithm as its proximal operator. The main application of this idea is to time-lagged regression, where we predict an outcome at time t from features at the previous K time points. In this setting, it is natural to assume that the coefficients decay as we move farther away from t, and hence the order constraint is reasonable. Potential application areas include financial time series and prediction of dynamic patient outcomes based on clinical measurements. We illustrate this idea on real and simulated data.

[1]  H. D. Brunk,et al.  Statistical inference under order restrictions : the theory and application of isotonic regression , 1973 .

[2]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[3]  B. Efron How Biased is the Apparent Error Rate of a Prediction Rule , 1986 .

[4]  Robert Tibshirani,et al.  Nearly-Isotonic Regression , 2011, Technometrics.

[5]  S. Pandey,et al.  What Are Degrees of Freedom , 2008 .

[6]  Jeffrey M. Wooldridge,et al.  Introductory Econometrics: A Modern Approach , 1999 .

[7]  D. Ruppert The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[8]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  R. Tibshirani,et al.  Degrees of freedom in lasso problems , 2011, 1111.0653.

[10]  J. Leeuw,et al.  Isotone Optimization in R: Pool-Adjacent-Violators Algorithm (PAVA) and Active Set Methods , 2009 .

[11]  C. Witzgall,et al.  Projections onto order simplexes , 1984 .

[12]  A. Fielding,et al.  Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression , 1974 .

[13]  H. Zou The Adaptive Lasso and Its Oracle Properties , 2006 .

[14]  R. Tibshirani,et al.  On the “degrees of freedom” of the lasso , 2007, 0712.0881.

[15]  R. Tibshirani,et al.  A LASSO FOR HIERARCHICAL INTERACTIONS. , 2012, Annals of statistics.

[16]  Y. Nardi,et al.  Autoregressive process modeling via the Lasso procedure , 2008, J. Multivar. Anal..

[17]  E. Makalic,et al.  Estimation of stationary autoregressive models with the Bayesian LASSO , 2013 .