Active set algorithms for isotonic regression; A unifying framework

AbstractIn this and subsequent papers we will show that several algorithms for the isotonic regression problem may be viewed as active set methods. The active set approach provides a unifying framework for studying algorithms for isotonic regression, simplifies the exposition of existing algorithms and leads to several new efficient algorithms. We also investigate the computational complexity of several algorithms.In this paper we consider the isotonic regression problem with respect to a complete order $$\begin{gathered} minimize\sum\limits_{i = 1}^n {w_i } (y_i - x_i )^2 \hfill \\ subject tox_1 \leqslant x_2 \leqslant \cdot \cdot \cdot \leqslant x_n \hfill \\ \end{gathered} $$ where eachwi is strictly positive and eachyi is an arbitrary real number. We show that the Pool Adjacent Violators algorithm (due to Ayer et al., 1955; Miles, 1959; Kruskal, 1964), is a dual feasible active set method and that the Minimum Lower Set algorithm (due to Brunk et al., 1957) is a primal feasible active set method of computational complexity O(n2). We present a new O(n) primal feasible active set algorithm. Finally we discuss Van Eeden's method and show that it is of worst-case exponential time complexity.

[1]  William L. Maxwell,et al.  Establishing Consistent and Realistic Reorder Intervals in Production-Distribution Systems , 1985, Oper. Res..

[2]  Ming S. Hung,et al.  Technical Note - A Polynomial Simplex Method for the Assignment Problem , 1983, Oper. Res..

[3]  Richard L. Dykstra An isotonic regression algorithm , 1981 .

[4]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[5]  Chu-in Charles Lee,et al.  The Min-Max Algorithm and Isotonic Regression , 1983 .

[6]  Friedrich Gebhardt,et al.  An algorithm for monotone regression with one or more independent variables , 1970 .

[7]  R. E. Miles THE COMPLETE AMALGAMATION INTO BLOCKS, BY WEIGHTED MEANS, OF A FINITE SET OF REAL NUMBERS , 1959 .

[8]  J. Orlin On the simplex algorithm for networks and generalized networks , 1983 .

[9]  J. Kruskal Nonmetric multidimensional scaling: A numerical method , 1964 .

[10]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[11]  F. T. Wright Estimating Strictly Increasing Regression Functions , 1978 .

[12]  Michel Balinski,et al.  A competitive (dual) simplex method for the assignment problem , 1986, Math. Program..

[13]  C. Witzgall,et al.  Projections onto order simplexes , 1984 .

[14]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[15]  H. D. Brunk,et al.  Minimizing integrals in certain classes of monotone functions. , 1957 .

[16]  Melanie L. Lenard,et al.  A computational study of active set strategies in nonlinear programming with linear constraints , 1979, Math. Program..

[17]  J. Kalbfleisch Statistical Inference Under Order Restrictions , 1975 .

[18]  Constance Van Eeden Maximum Likelihood Estimation Of Ordered Probabilities1) , 1956 .

[19]  Robin Roundy,et al.  A 98%-Effective Lot-Sizing Rule for a Multi-Product, Multi-Stage Production / Inventory System , 1986, Math. Oper. Res..

[20]  Michael J. Best,et al.  A quadratic programming algorithm , 1988, ZOR Methods Model. Oper. Res..

[21]  Philip E. Gill,et al.  Practical optimization , 1981 .

[22]  Michael J. Best,et al.  Equivalence of some quadratic programming algorithms , 1984, Math. Program..