Free-Steering Relaxation Methods for Problems with Strictly Convex Costs and Linear Constraints

We consider dual coordinate ascent methods for minimizing a strictly convex possibly nondifferentiable function subject to linear constraints. Such methods are useful in large-scale applications e.g., entropy maximization, quadratic programming, network flow, because they are simply, can exploit sparsity and in certain cases are highly parallelizable. We establish their global convergence under weak conditions and a free-steering order of relaxation. Previous comparable results were restricted to special problems with separable costs and equality constraints. Our convergence framework unifies to a certain extent the approaches of Bregman, Censor and Lent, De Pierro and Iusem, and Luo and Tseng, and complements that of Bertsekas and Tseng.

[1]  Yair Censor,et al.  Parallel application of block-iterative methods in medical imaging and radiation therapy , 1988, Math. Program..

[2]  D. Bertsekas,et al.  Relaxation methods for network flow problems with convex arc costs , 1987 .

[3]  K. Kiwiel Proximal Minimization Methods with Generalized Bregman Functions , 1997 .

[4]  Marc Teboulle,et al.  Convergence Analysis of a Proximal-Like Minimization Algorithm Using Bregman Functions , 1993, SIAM J. Optim..

[5]  Y. Censor,et al.  On iterative methods for linearly constrained entropy maximization , 1990 .

[6]  Tommy Elfving,et al.  An algorithm for maximum entropy image reconstruction from noisy data , 1989 .

[7]  Sven Erlander,et al.  Entropy in linear programs , 1981, Math. Program..

[8]  Y. Censor Row-Action Methods for Huge and Sparse Systems and Their Applications , 1981 .

[9]  Y. Censor,et al.  Parallel computing with block-iterative image reconstruction algorithms , 1991 .

[10]  Y. Censor,et al.  Proximal minimization algorithm withD-functions , 1992 .

[11]  A. Barrett Network Flows and Monotropic Optimization. , 1984 .

[12]  Paul Tseng,et al.  Error Bound and Reduced-Gradient Projection Algorithms for Convex Minimization over a Polyhedral Set , 1993, SIAM J. Optim..

[13]  Clifford Hildreth,et al.  A quadratic programming procedure , 1957 .

[14]  A. Pierro,et al.  A relaxed version of Bregman's method for convex programming , 1986 .

[15]  J. E. Falk Lagrange multipliers and nonlinear programming , 1967 .

[16]  Yair Censor,et al.  Interval-constrained matrix balancing , 1991 .

[17]  N. F. Stewart,et al.  Bregman's balancing method , 1981 .

[18]  Alfredo N. Iusem On Dual Convergence and the Rate of Primal Convergence of Bregman's Convex Programming Method , 1991, SIAM J. Optim..

[19]  Y. Censor,et al.  Optimization of “log x entropy over linear equality constraints , 1987 .

[20]  Claude Lemaréchal,et al.  Global and superlinear convergence of an algorithm for one-dimensional minimization of convex functions , 1982, Math. Program..

[21]  Y. Censor,et al.  An interior points algorithm for the convex feasibility problem , 1983 .

[22]  P. Tseng,et al.  On the convergence of the coordinate descent method for convex differentiable minimization , 1992 .

[23]  Robert Mifflin An implementation of an algorithm for univariate minimization and an application to nested optimization , 1988, Math. Program..

[24]  T. Elfving On some methods for entropy maximization and matrix scaling , 1980 .

[25]  N. Ottavy Strong convergence of projection-like methods in Hilbert spaces , 1988 .

[26]  Heinz H. Bauschke,et al.  Legendre functions and the method of random Bregman projections , 1997 .

[27]  P. Tseng Descent methods for convex essentially smooth minimization , 1991 .

[28]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[29]  Stavros A. Zenios,et al.  Massively Parallel Algorithms for Singly Constrained Convex Programs , 1992, INFORMS J. Comput..

[30]  K. Kiwiel Block-iterative surrogate projection methods for convex feasibility problems , 1995 .

[31]  Stavros A. Zenios,et al.  On the Fine-Grain Decomposition of Multicommodity Transportation Problems , 1991, SIAM J. Optim..

[32]  Y. Censor,et al.  On some optimization techniques in image reconstruction from projections , 1987 .

[33]  Y. Censor,et al.  Optimization of Burg's entropy over linear constraints , 1991 .

[34]  Olvi L. Mangasarian Sparsity-preserving sor algorithms for separable quadratic and linear programming , 1984, Comput. Oper. Res..

[35]  J. Darroch,et al.  Generalized Iterative Scaling for Log-Linear Models , 1972 .

[36]  Paul Tseng,et al.  Error Bound and Convergence Analysis of Matrix Splitting Algorithms for the Affine Variational Inequality Problem , 1992, SIAM J. Optim..

[37]  P. Tseng,et al.  On the linear convergence of descent methods for convex essentially smooth minimization , 1992 .

[38]  P. Tseng Dual ascent methods for problems with strictly convex costs and linear constraints: a unified approach , 1990 .

[39]  Paul Tseng,et al.  Relaxation Methods for Problems with Strictly Convex Costs and Linear Constraints , 1991, Math. Oper. Res..

[40]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[41]  Stavros A. Zenios,et al.  A Massively Parallel Algorithm for Nonlinear Stochastic Network Problems , 1993, Oper. Res..

[42]  Y. Censor,et al.  An iterative row-action method for interval convex programming , 1981 .

[43]  Marc Teboulle,et al.  Entropic Proximal Mappings with Applications to Nonlinear Programming , 1992, Math. Oper. Res..

[44]  Jonathan Eckstein,et al.  Nonlinear Proximal Point Algorithms Using Bregman Functions, with Applications to Convex Programming , 1993, Math. Oper. Res..

[45]  Stavros A. Zenios,et al.  Proximal minimizations with D-functions and the massively parallel solution of linear network programs , 1993, Comput. Optim. Appl..

[46]  P. Tseng,et al.  On the convergence of a matrix splitting algorithm for the symmetric monotone linear complementarity problem , 1991 .

[47]  Paul Tseng,et al.  On the Convergence Rate of Dual Ascent Methods for Linearly Constrained Convex Minimization , 1993, Math. Oper. Res..

[48]  P. J. Chase,et al.  Order independence and factor convergence in iterative scaling , 1993 .

[49]  Paul Tseng,et al.  Relaxation methods for problems with strictly convex separable costs and linear constraints , 1987, Math. Program..

[50]  J. Pang,et al.  Iterative methods for large convex quadratic programs: a survey , 1987 .

[51]  Yair Censor,et al.  Massively Parallel Row-Action Algorithms for Some Nonlinear Transportation Problems , 1991, SIAM J. Optim..

[52]  Gabor T. Herman,et al.  A family of iterative quadratic optimization algorithms for pairs of inequalities, with application in diagnostic radiology , 1978 .

[53]  J. Zowe,et al.  Relaxed outer projections, weighted averages and convex feasibility , 1990 .