Tackling Box-Constrained Optimization via a New Projected Quasi-Newton Approach

Numerous scientific applications across a variety of fields depend on box-constrained convex optimization. Box-constrained problems therefore continue to attract research interest. We address box-constrained (strictly convex) problems by deriving two new quasi-Newton algorithms. Our algorithms are positioned between the projected-gradient [J. B. Rosen, J. SIAM, 8 (1960), pp. 181-217] and projected-Newton [D. P. Bertsekas, SIAM J. Control Optim., 20 (1982), pp. 221-246] methods. We also prove their convergence under a simple Armijo step-size rule. We provide experimental results for two particular box-constrained problems: nonnegative least squares (NNLS), and nonnegative Kullback-Leibler (NNKL) minimization. For both NNLS and NNKL our algorithms perform competitively as compared to well-established methods on medium-sized problems; for larger problems our approach frequently outperforms the competition.

[1]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[2]  Robert M. Lewitt,et al.  Overview of methods for image reconstruction from projections in emission computed tomography , 2003, Proc. IEEE.

[3]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[4]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[5]  H. Malcolm Hudson,et al.  Accelerated image reconstruction using ordered subsets of projection data , 1994, IEEE Trans. Medical Imaging.

[6]  J. Nagy,et al.  Enforcing nonnegativity in image reconstruction algorithms , 2000, SPIE Optics + Photonics.

[7]  J. B. Rosen.  The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints , 1960 .

[8]  J. Dunn.  Global and Asymptotic Convergence Rate Estimates for a Class of Projected Gradient Processes , 1981 .

[9]  李幼升,et al.  Ph , 1989 .

[10]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[11]  R. Bro,et al.  A fast non‐negativity‐constrained least squares algorithm , 1997 .

[12]  Daniel D. Lee,et al.  Statistical signal processing with nonnegativity constraints , 2003, INTERSPEECH.

[13]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[14]  Nicolas Molinari,et al.  Bounded optimal knots for regression splines , 2004, Comput. Stat. Data Anal..

[15]  Brad Schofield.  Vehicle Dynamics Control for Rollover Prevention , 2006 .

[16]  M. Bierlaire,et al.  On iterative algorithms for linear least squares problems with bound constraints , 1991 .

[17]  J. B. Rosen.  The gradient projection method for nonlinear programming: Part II , 1961 .

[18]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[19]  Yin Zhang,et al.  Interior-Point Gradient Method for Large-Scale Totally Nonnegative Least Squares Problems , 2005 .

[20]  Philip E. Gill,et al.  Practical optimization , 1981 .