ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems

This paper concerns multiobjective optimization in scenarios where each solution evaluation is financially and/or temporally expensive. We make use of nine relatively low-dimensional, nonpathological, real-valued functions, such as arise in many applications, and assess the performance of two algorithms after just 100 and 250 (or 260) function evaluations. The results show that NSGA-II, a popular multiobjective evolutionary algorithm, performs well compared with random search, even within the restricted number of evaluations used. A significantly better performance (particularly, in the worst case) is, however, achieved on our test set by an algorithm proposed herein-ParEGO-which is an extension of the single-objective efficient global optimization (EGO) algorithm of Jones et al. ParEGO uses a design-of-experiments inspired initialization procedure and learns a Gaussian processes model of the search landscape, which is updated after every function evaluation. Overall, ParEGO exhibits a promising performance for multiobjective optimization problems where evaluations are expensive or otherwise restricted in number.

[1]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[2]  William H. Press,et al.  Book-Review - Numerical Recipes in Pascal - the Art of Scientific Computing , 1989 .

[3]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[4]  William H. Press,et al.  Numerical recipes in C++: the art of scientific computing, 2nd Edition (C++ ed., print. is corrected to software version 2.10) , 1994 .

[5]  Jeffrey Horn,et al.  Multiobjective Optimization Using the Niched Pareto Genetic Algorithm , 1993 .

[6]  C. Wandrey,et al.  Medium Optimization by Genetic Algorithm for Continuous Production of Formate Dehydrogenase , 1995 .

[7]  Hans-Paul Schwefel,et al.  Parallel Problem Solving from Nature — PPSN IV , 1996, Lecture Notes in Computer Science.

[8]  San Cristóbal Mateo,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996 .

[9]  Peter J. Fleming,et al.  On the Performance Assessment and Comparison of Stochastic Multiobjective Optimizers , 1996, PPSN.

[10]  Alain Ratle,et al.  Accelerating the Convergence of Evolutionary Algorithms by Fitness Landscape Approximation , 1998, PPSN.

[11]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[12]  Kalyanmoy Deb,et al.  Evolutionary Algorithms for Multi-Criterion Optimization in Engineering Design , 1999 .

[13]  D. Quagliarella,et al.  Airfoil and wing design through hybrid optimization strategies , 1998 .

[14]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithm test suites , 1999, SAC '99.

[15]  Eckart Zitzler,et al.  Evolutionary algorithms for multiobjective optimization: methods and applications , 1999 .

[16]  R. J. Gilbert,et al.  Efficient Improvement of Silage Additives by Using Genetic Algorithms , 2000, Applied and Environmental Microbiology.

[17]  Carlos A. Coello Coello,et al.  An updated survey of GA-based multiobjective optimization techniques , 2000, CSUR.

[18]  X. Yao Evolutionary Search of Approximated N-dimensional Landscapes , 2000 .

[19]  John W. Hartmann,et al.  Optimal multi-objective low-thrust spacecraft trajectories , 2000 .

[20]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[21]  Kalyanmoy Deb,et al.  Self-Adaptive Genetic Algorithms with Simulated Binary Crossover , 2001, Evolutionary Computation.

[22]  Carlos A. Coello Coello,et al.  A Micro-Genetic Algorithm for Multiobjective Optimization , 2001, EMO.

[23]  P. Coveney,et al.  Combinatorial searches of inorganic materials using the ink-jet printer: science, philosophy and technology , 2001 .

[24]  Martin J. Oates,et al.  PESA-II: region-based selection in evolutionary multiobjective optimization , 2001 .

[25]  Petros Koumoutsakos,et al.  Self-organizing Maps for Pareto Optimization of Airfoils , 2002, PPSN.

[26]  Joshua D. Knowles Local-search and hybrid evolutionary algorithms for Pareto optimization , 2002 .

[27]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[28]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[29]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[30]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[31]  Kyriakos C. Giannakoglou,et al.  Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence , 2002 .

[32]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[33]  Thomas Bäck,et al.  Evaluating Multi-criteria Evolutionary Algorithms for Airfoil Optimisation , 2002, PPSN.

[34]  Martin J. Oates,et al.  Landscape State Machines: Tools for Evolutionary Algorithm Performance Analyses and Landscape/Algorithm Mapping , 2003, EvoWorkshops.

[35]  Waldo Gonzalo Cancino Ticona,et al.  Multiobjective Evolutionary Algorithms Applied to the Rehabilitation of a Water Distribution System: A Comparative Study , 2003, EMO.

[36]  Mikkel T. Jensen,et al.  Reducing the run-time complexity of multiobjective EAs: The NSGA-II and other algorithms , 2003, IEEE Trans. Evol. Comput..

[37]  D. Kell,et al.  Explanatory optimization of protein mass spectrometry via genetic search. , 2003, Analytical chemistry.

[38]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[39]  Evan J. Hughes Multi-objective Binary Search Optimisation , 2003, EMO.

[40]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[41]  D. Kell,et al.  Selective detection of proteins in mixtures using electrospray ionization mass spectrometry: influence of instrumental settings and implications for proteomics. , 2004, Analytical chemistry.

[42]  António Gaspar-Cunha,et al.  A Hybrid Multi-Objective Evolutionary Algorithm Using an Inverse Neural Network , 2004, Hybrid Metaheuristics.

[43]  Oliver Fiehn,et al.  Faculty Opinions recommendation of Explanatory optimization of protein mass spectrometry via genetic search. , 2004 .

[44]  Michael Emmerich,et al.  Metamodel Assisted Multiobjective Optimisation Strategies and their Application in Airfoil Design , 2004 .

[45]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[46]  Bernhard Sendhoff,et al.  On Test Functions for Evolutionary Multi-objective Optimization , 2004, PPSN.

[47]  Jonathan E. Fieldsend,et al.  Dominance measures for multi-objective simulated annealing , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[48]  Christine A. Shoemaker,et al.  Local function approximation in evolutionary algorithms for the optimization of costly functions , 2004, IEEE Transactions on Evolutionary Computation.

[49]  Kalyanmoy Deb,et al.  Dynamic multiobjective optimization problems: test cases, approximations, and applications , 2004, IEEE Transactions on Evolutionary Computation.

[50]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[51]  Douglas B. Kell,et al.  A metabolome pipeline: from concept to data to knowledge , 2005, Metabolomics.

[52]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[53]  Kok Wai Wong,et al.  Surrogate-Assisted Evolutionary Optimization Frameworks for High-Fidelity Engineering Design Problems , 2005 .

[54]  Nicola Beume,et al.  An EMO Algorithm Using the Hypervolume Measure as Selection Criterion , 2005, EMO.

[55]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[56]  Joshua D. Knowles,et al.  Closed-loop, multiobjective optimization of analytical instrumentation: gas chromatography/time-of-flight mass spectrometry of the metabolomes of human serum and of yeast fermentations. , 2005, Analytical chemistry.

[57]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..