Truncated-newtono algorithms for large-scale unconstrained optimization

We present an algorithm for large-scale unconstrained optimization based onNewton's method. In large-scale optimization, solving the Newton equations at each iteration can be expensive and may not be justified when far from a solution. Instead, an inaccurate solution to the Newton equations is computed using a conjugate gradient method. The resulting algorithm is shown to have strong convergence properties and has the unusual feature that the asymptotic convergence rate is a user specified parameter which can be set to anything between linear and quadratic convergence. Some numerical results on a 916 vriable test problem are given. Finally, we contrast the computational behavior of our algorithm with Newton's method and that of a nonlinear conjugate gradient algorithm.

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  D. Luenberger Hyperbolic Pairs in the Method of Conjugate Gradients , 1969 .

[3]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[4]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[5]  M. Powell,et al.  On the Estimation of Sparse Jacobian Matrices , 1974 .

[6]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[7]  R. Fletcher Conjugate gradient methods for indefinite systems , 1976 .

[8]  O. Axelsson Solution of linear systems of equations: Iterative methods , 1977 .

[9]  S. Eisenstat,et al.  The Modified Conjugate Residual Method for Partial Differential Equations. , 1977 .

[10]  D. Shanno On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[11]  P. Toint Some numerical results using a sparse matrix updating formula in unconstrained optimization , 1978 .

[12]  R. Tapia,et al.  QDN: a variable storage algorithm for unconstrained optimization , 1978 .

[13]  R. Chandra Conjugate gradient methods for partial differential equations. , 1978 .

[14]  Michael A. Saunders,et al.  Large-scale linearly constrained optimization , 1978, Math. Program..

[15]  M. Powell,et al.  On the Estimation of Sparse Hessian Matrices , 1979 .

[16]  David F. Shanno,et al.  Remark on “Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4]” , 1980, TOMS.

[17]  Magnus R. Hestenes,et al.  Conjugate Direction Methods in Optimization , 1980 .

[18]  J. Nocedal Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[19]  R. Dembo,et al.  A scaled reduced gradient algorithm for network flow problems with convex separable costs , 1981 .

[20]  Philip E. Gill,et al.  Practical optimization , 1981 .

[21]  Dianne P. O'Leary,et al.  A discrete Newton algorithm for minimizing a function of many variables , 1982, Math. Program..

[22]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[23]  John D. Ramsdell,et al.  Estimation of Sparse Jacobian Matrices , 1983 .