Parallel Space Decomposition of the Mesh Adaptive Direct Search Algorithm

This paper describes a parallel space decomposition PSD technique for the mesh adaptive direct search MADS algorithm. MADS extends a generalized pattern search for constrained nonsmooth optimization problems. The objective of the present work is to obtain good solutions to larger problems than the ones typically solved by MADS. The new method PSD-MADS is an asynchronous parallel algorithm in which the processes solve problems over subsets of variables. The convergence analysis based on the Clarke calculus is essentially the same as for the MADS algorithm. A practical implementation is described, and some numerical results on problems with up to 500 variables illustrate the advantages and limitations of PSD-MADS.

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[3]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[4]  John E. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991, SIAM J. Optim..

[5]  Paul Tseng,et al.  Dual coordinate ascent methods for non-strictly convex minimization , 1993, Math. Program..

[6]  Boxin Tang.  Orthogonal Array-Based Latin Hypercubes , 1993 .

[7]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[8]  Michael C. Ferris,et al.  Parallel Variable Distribution , 1994, SIAM J. Optim..

[9]  O. Mangasarian.  Parallel Gradient Distribution in Unconstrained Optimization , 1995 .

[10]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[11]  M. Solodov.  New Inexact Parallel Variable Distribution Algorithms , 1997, Comput. Optim. Appl..

[12]  Masao Fukushima,et al.  Parallel Variable Transformation in Unconstrained Optimization , 1998, SIAM J. Optim..

[13]  John E. Dennis,et al.  Managing surrogate objectives to optimize a helicopter rotor design - Further experiments , 1998 .

[14]  Mikhail V. Solodov,et al.  On the Convergence of Constrained Parallel Variable Distribution Algorithms , 1998, SIAM J. Optim..

[15]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[16]  A. Frommer,et al.  A unified approach to parallel space decomposition methods , 1999 .

[17]  Robert Michael Lewis,et al.  Pattern Search Algorithms for Bound Constrained Minimization , 1999, SIAM J. Optim..

[18]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[19]  Masao Fukushima,et al.  Testing Parallel Variable Transformation , 1999, Comput. Optim. Appl..

[20]  Robert Michael Lewis,et al.  Pattern Search Methods for Linearly Constrained Minimization , 1999, SIAM J. Optim..

[21]  Chin-Sung Liu,et al.  Parallel Synchronous and Asynchronous Space-Decomposition Algorithms for Large-Scale Minimization Problems , 2000, Comput. Optim. Appl..

[22]  I. Coope,et al.  Frame Based Methods for Unconstrained Optimization , 2000 .

[23]  V. Torczon,et al.  Direct search methods: then and now , 2000 .

[24]  Tamara G. Kolda,et al.  Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2001, SIAM J. Sci. Comput..

[25]  J. Dennis,et al.  Mixed Variable Optimization of the Number and Composition of Heat Intercepts in a Thermal Insulation System , 2001 .

[26]  C. T. Kelley,et al.  A Locally-Biased form of the DIRECT Algorithm , 2001, J. Glob. Optim..

[27]  Charles Audet,et al.  Catalytic Combustion Kinetics: Using a Direct Search Algorithm to Evaluate Kinetic Parameters from Light-Off Curves , 2002 .

[28]  Charles Audet,et al.  Analysis of Generalized Pattern Searches , 2000, SIAM J. Optim..

[29]  Claudia A. Sagastizábal,et al.  Parallel Variable Distribution for Constrained Optimization , 2002, Comput. Optim. Appl..

[30]  Zhijun Wu,et al.  Parallel continuous optimization , 2003 .

[31]  Tamara G. Kolda,et al.  Understanding Asynchronous Parallel Pattern Search , 2003 .

[32]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[33]  Meng Wang,et al.  Optimal Aeroacoustic Shape Design Using the Surrogate Management Framework , 2003 .

[34]  C. J. Price,et al.  Frames and Grids in Unconstrained and Linearly Constrained Optimization: A Nonsmooth Approach , 2003, SIAM J. Optim..

[35]  Cass T. Miller,et al.  Solution of a Well-Field Design Problem with Implicit Filtering , 2004 .

[36]  Tamara G. Kolda,et al.  On the Convergence of Asynchronous Parallel Pattern Search , 2002, SIAM J. Optim..

[37]  Fernando Nogueira,et al.  Pattern Search Methods for User-Provided Points: Application to Molecular Geometry Problems , 2004, SIAM J. Optim..

[38]  Charles Audet,et al.  Finding Optimal Algorithmic Parameters Using the Mesh Adaptive Direct Search Algorithm , 2004 .

[39]  Sandia Report,et al.  Revisiting Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2004 .

[40]  D. Finkel,et al.  Convergence analysis of the direct algorithm , 2004 .

[41]  M. Abramson.  Mixed Variable Optimization of a Load-Bearing Thermal Insulation System Using a Filter Pattern Search Algorithm , 2004 .

[42]  Tamara G. Kolda,et al.  Revisiting Asynchronous Parallel Pattern Search for Nonlinear Optimization , 2005, SIAM J. Optim..

[43]  Charles Audet,et al.  Second-Order Convergence of Mesh-Adaptive Direct Search , 2005 .

[44]  Masao Fukushima,et al.  Derivative-Free Filter Simulated Annealing Method for Constrained Continuous Global Optimization , 2006, J. Glob. Optim..

[45]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[46]  CHARLES AUDET,et al.  Finding Optimal Algorithmic Parameters Using Derivative-Free Optimization , 2006, SIAM J. Optim..

[47]  Tamara G. Kolda,et al.  Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivative-free optimization , 2006, TOMS.

[48]  Charles Audet,et al.  Convergence of Mesh Adaptive Direct Search to Second-Order Stationary Points , 2006, SIAM J. Optim..

[49]  John E. Dennis,et al.  MoVars: Multidisciplinary Optimization Via Adaptive Response Surfaces , 2007 .

[50]  Charles Audet,et al.  Erratum: Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[51]  Charles Audet,et al.  Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search , 2006, J. Glob. Optim..

[52]  Charles Audet,et al.  A MADS Algorithm with a Progressive Barrier for Derivative-Free Nonlinear Programming , 2007 .