Conjugate Gradient Methods with Inexact Searches

Conjugate gradient methods are iterative methods for finding the minimizer of a scalar function fx of a vector variable x which do not update an approximation to the inverse Hessian matrix. This paper examines the effects of inexact linear searches on the methods and shows how the traditional Fletcher-Reeves and Polak-Ribiere algorithm may be modified in a form discovered by Perry to a sequence which can be interpreted as a memorytess BFGS algorithm. This algorithm may then be scaled optimally in the sense of Oren and Spedicalo. This scaling can be combined with Beale restarts and Powell's restart criterion. Computational results will show that this new method substantially outperforms known conjugate gradient methods on a wide class of problems.

[1]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[2]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[3]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[4]  Garth P. McCormick,et al.  Methods of conjugate directions versus quasi-Newton methods , 1972, Math. Program..

[5]  H. Crowder,et al.  Linear convergence of the conjugate gradient method , 1972 .

[6]  Melanie L. Lenard,et al.  Practical convergence conditions for unconstrained optimization , 1973, Math. Program..

[7]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[8]  L. Dixon.  Conjugate Gradient Algorithms: Quadratic Termination without Linear Searches , 1975 .

[9]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[10]  Shmuel S. Oren,et al.  Optimal conditioning of self-scaling variable Metric algorithms , 1976, Math. Program..

[11]  A. Perry.  A Modified Conjugate Gradient Algorithm for Unconstrained Nonlinear Optimization , 1975 .

[12]  M. J. D. Powell,et al.  Some convergence properties of the conjugate gradient method , 1976, Math. Program..

[13]  L. Nazareth.  Relationship between the BFGS and conjugate gradient algorithms , 1977 .

[14]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[15]  A. Perry.  A Class of Conjugate Gradient Algorithms with a Two-Step Variable Metric Memory , 1977 .

[16]  Ken Brodlie,et al.  An assessment of two approaches to variable metric methods , 1977, Math. Program..

[17]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[18]  D. F. Shanno,et al.  Matrix conditioning and nonlinear optimization , 1978, Math. Program..