Nearly-Isotonic Regression

We consider the problem of approximating a sequence of data points with a “nearly-isotonic,” or nearly-monotone function. This is formulated as a convex optimization problem that yields a family of solutions, with one extreme member being the standard isotonic regression fit. We devise a simple algorithm to solve for the path of solutions, which can be viewed as a modified version of the well-known pool adjacent violators algorithm, and computes the entire path in O(n log n) operations (n being the number of data points). In practice, the intermediate fits can be used to examine the assumption of monotonicity. Nearly-isotonic regression admits a nice property in terms of its degrees of freedom: at any point along the path, the number of joined pieces in the solution is an unbiased estimate of its degrees of freedom. We also extend the ideas to provide “nearly-convex” approximations.

[1]  Holger Hoefling.  A Path Algorithm for the Fused Lasso Signal Approximator , 2009, 0910.0526.

[2]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[3]  D. J. Bartholomew,et al.  A TEST OF HOMOGENEITY FOR ORDERED ALTERNATIVES. II , 1959 .

[4]  R. E. Miles.  THE COMPLETE AMALGAMATION INTO BLOCKS, BY WEIGHTED MEANS, OF A FINITE SET OF REAL NUMBERS , 1959 .

[5]  R. Tibshirani,et al.  The solution path of the generalized lasso , 2010, 1005.1971.

[6]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[7]  C. Witzgall,et al.  Projections onto order simplexes , 1984 .

[8]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[9]  H. D. Brunk.  Maximum Likelihood Estimates of Monotone Parameters , 1955 .

[10]  J. Leeuw,et al.  Isotone Optimization in R: Pool-Adjacent-Violators Algorithm (PAVA) and Active Set Methods , 2009 .

[11]  M. Woodroofe,et al.  Isotonic regression: Another look at the changepoint problem , 2001 .

[12]  C. Stein.  Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[13]  Mary C. Meyer,et al.  ON THE DEGREES OF FREEDOM IN SHAPE-RESTRICTED REGRESSION , 2000 .

[14]  S. Pandey,et al.  What Are Degrees of Freedom , 2008 .

[15]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[16]  B. Efron.  How Biased is the Apparent Error Rate of a Prediction Rule , 1986 .

[17]  L. Evans.  Measure theory and fine properties of functions , 1992 .

[18]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[19]  H. D. Brunk,et al.  Statistical inference under order restrictions : the theory and application of isotonic regression , 1973 .

[20]  P. Schlegel,et al.  The explicit inverse of a tridiagonal matrix , 1970 .

[21]  A. Fielding,et al.  Statistical Inference Under Order Restrictions. The Theory and Application of Isotonic Regression , 1974 .