Parallel Multi-Objective Evolutionary Algorithms: A Comprehensive Survey

Abstract Multi-Objective Evolutionary Algorithms (MOEAs) are powerful search techniques that have been extensively used to solve difficult problems in a wide variety of disciplines. However, they can be very demanding in terms of computational resources. Parallel implementations of MOEAs (pMOEAs) provide considerable gains regarding performance and scalability and, therefore, their relevance in tackling computationally expensive applications. This paper presents a survey of pMOEAs, describing a refined taxonomy, an up-to-date review of methods and the key contributions to the field. Furthermore, some of the open questions that require further research are also briefly discussed.

[1]  Xin Yao,et al.  A benchmark test suite for evolutionary many-objective optimization , 2017 .

[2]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[3]  J. Vrugt,et al.  Inverse Modeling of Subsurface Flow and Transport Properties: A Review with New Developments , 2008 .

[4]  Carlos A. Coello Coello,et al.  A Study of the Parallelization of a Coevolutionary Multi-objective Evolutionary Algorithm , 2004, MICAI.

[5]  Selim G. Akl,et al.  Design and analysis of parallel algorithms , 1985 .

[6]  Mariem Gzara,et al.  Parallel Multi-Objective Evolutionary Algorithm with Multi-Front Equitable Distribution , 2006, 2006 Fifth International Conference on Grid and Cooperative Computing (GCC'06).

[7]  Aurora Trinidad Ramirez Pozo,et al.  Parallel MOEA/D-ACO on GPU , 2014, IBERAMIA.

[8]  Pascal Bouvry,et al.  A parallel cooperative coevolutionary SMPSO algorithm for multi-objective optimization , 2016, 2016 International Conference on High Performance Computing & Simulation (HPCS).

[9]  Kyriakos C. Giannakoglou,et al.  A grid-enabled asynchronous metamodel-assisted evolutionary algorithm for aerodynamic optimization , 2009, Genetic Programming and Evolvable Machines.

[10]  Nedjmeddine Kantour,et al.  A parallel MOEA with criterion-based selection applied to the Knapsack Problem , 2019 .

[11]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[12]  Radek Hrbacek,et al.  Parallel Multi-Objective Evolutionary Design of Approximate Circuits , 2015, GECCO.

[13]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for solving multiobjective optimization problems , 2006, Int. J. Intell. Syst..

[14]  Shengxiang Yang,et al.  A Grid-Based Evolutionary Algorithm for Many-Objective Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[15]  Regina Berretta,et al.  PasMoQAP: A parallel asynchronous memetic algorithm for solving the Multi-Objective Quadratic Assignment Problem , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[16]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[17]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[18]  Xin Yao,et al.  Many-Objective Evolutionary Algorithms , 2015, ACM Comput. Surv..

[19]  Nicola Beume,et al.  An EMO Algorithm Using the Hypervolume Measure as Selection Criterion , 2005, EMO.

[20]  Zhi-hui Zhan,et al.  Parallel multi-strategy evolutionary algorithm using massage passing interface for many-objective optimization , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[21]  Jun Zhang,et al.  An Evolutionary Algorithm with Double-Level Archives for Multiobjective Optimization , 2015, IEEE Transactions on Cybernetics.

[22]  Julio Ortega Lopera,et al.  Comparison of parallel multi-objective approaches to protein structure prediction , 2011, The Journal of Supercomputing.

[23]  Marc Schoenauer,et al.  Asynchronous master/slave moeas and heterogeneous evaluation costs , 2012, GECCO '12.

[24]  Dario Izzo,et al.  PaDe: A Parallel Algorithm Based on the MOEA/D Framework and the Island Model , 2014, PPSN.

[25]  Witold Pedrycz,et al.  PEA: Parallel Evolutionary Algorithm by Separating Convergence and Diversity for Large-Scale Multi-Objective Optimization , 2018, 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS).

[26]  Heike Trautmann,et al.  On the properties of the R2 indicator , 2012, GECCO '12.

[27]  Anne Auger,et al.  Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point , 2009, FOGA '09.

[28]  Enrique Alba,et al.  A Parallel Version of SMS-EMOA for Many-Objective Optimization Problems , 2016, PPSN.

[29]  Carlos A. Coello Coello,et al.  MRMOGA: parallel evolutionary multiobjective optimization using multiple resolutions , 2005, 2005 IEEE Congress on Evolutionary Computation.

[30]  Hisao Ishibuchi,et al.  On the Effect of the Cooperation of Indicator-Based Multiobjective Evolutionary Algorithms , 2021, IEEE Transactions on Evolutionary Computation.

[31]  Francisco Luna,et al.  Optimizing the DFCN Broadcast Protocol with a Parallel Cooperative Strategy of Multi-Objective Evolutionary Algorithms , 2009, EMO.

[32]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[33]  Minghan Li,et al.  A cooperative co-evolutionary algorithm for large-scale multi-objective optimization problems , 2018, GECCO.

[34]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[35]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[36]  El-Ghazali Talbi,et al.  Parallel Multi-Objective Approaches for Inferring Phylogenies , 2010, EvoBIO.

[37]  Günter Rudolph,et al.  Parallel Approaches for Multiobjective Optimization , 2008, Multiobjective Optimization.

[38]  Tobias Glasmachers.  Optimized Approximation Sets for Low-Dimensional Benchmark Pareto Fronts , 2014, PPSN.

[39]  Antonio J. Nebro,et al.  A Study of the Parallelization of the Multi-Objective Metaheuristic MOEA/D , 2010, LION.

[40]  Hong-Fei Teng,et al.  A Non-Dominated Sorting Cooperative Co-Evolutionary Differential Evolution Algorithm for Multi-Objective Layout Optimization , 2017, IEEE Access.

[41]  Gang Ju,et al.  A parallel genetic algorithm in multi-objective optimization , 2009, 2009 Chinese Control and Decision Conference.

[42]  Keiki Takadama,et al.  A study of self-adaptive semi-asynchronous evolutionary algorithm on multi-objective optimization problem , 2017, GECCO.

[43]  Gang Ju,et al.  A selective migration parallel multi-objective genetic algorithm , 2010, 2010 Chinese Control and Decision Conference.

[44]  Ewa Szlachcic,et al.  Parallel Distributed Genetic Algorithm for Expensive Multi-Objective Optimization Problems , 2009, EUROCAST.

[45]  Man Leung Wong,et al.  Data mining using parallel Multi-Objective Evolutionary algorithms on graphics hardware , 2010, IEEE Congress on Evolutionary Computation.

[46]  Carlos A. Coello Coello,et al.  Use of cooperative coevolution for solving large scale multiobjective optimization problems , 2013, 2013 IEEE Congress on Evolutionary Computation.

[47]  Anastasios S. Lyrintzis,et al.  AERODYNAMIC AND AERO ACOUSTIC OPTIMIZATION OF AIRFOILS VIA A PARALLEL GENETIC ALGORITHM , 1998 .

[48]  Hussein A. Abbass,et al.  MOCCA-II: A multi-objective co-operative co-evolutionary algorithm , 2014, Appl. Soft Comput..

[49]  Kai Xu,et al.  A scalable parallel genetic algorithm for x-ray spectroscopic analysis , 2005, GECCO '05.

[50]  Sébastien Vérel,et al.  A study on population size and selection lapse in many-objective optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[51]  Enrique Alba,et al.  A cellular multi-objective genetic algorithm for optimal broadcasting strategy in metropolitan MANETs , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[52]  Kalyanmoy Deb,et al.  Parallelizing multi-objective evolutionary algorithms: cone separation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[53]  Dipti Srinivasan,et al.  A Survey of Multiobjective Evolutionary Algorithms Based on Decomposition , 2017, IEEE Transactions on Evolutionary Computation.

[54]  Carlos A. Coello Coello,et al.  IGD+-EMOA: A multi-objective evolutionary algorithm based on IGD+ , 2016, CEC.

[55]  Andreas Zell,et al.  Parallelization of Multi-objective Evolutionary Algorithms Using Clustering Algorithms , 2005, EMO.

[56]  Carlos A. Coello Coello,et al.  Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multiobjective Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[57]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[58]  Man Leung Wong,et al.  Parallel multi-objective evolutionary algorithms on graphics processing units , 2009, GECCO '09.

[59]  Ofer M. Shir,et al.  A Reduced-Cost SMS-EMOA Using Kriging, Self-Adaptation, and Parallelization , 2008, MCDM.

[60]  Yalan Zhou,et al.  Ensemble of many-objective evolutionary algorithms for many-objective problems , 2017, Soft Comput..

[61]  Hisao Ishibuchi,et al.  Performance of Decomposition-Based Many-Objective Algorithms Strongly Depends on Pareto Front Shapes , 2017, IEEE Transactions on Evolutionary Computation.

[62]  J. Qiang,et al.  A PARALLEL MULTI-OBJECTIVE DIFFERENTIAL EVOLUTION ALGORITHM FOR PHOTOINJECTOR BEAM DYNAMICS OPTIMIZATION , 2013 .

[63]  J. Periaux,et al.  Parallel Genetic Solution for Multiobjective MDO , 1996, Parallel CFD.

[64]  Yuan Zhao,et al.  A distributed pool architecture for genetic algorithms , 2009, 2009 IEEE Congress on Evolutionary Computation.

[65]  Michael T. M. Emmerich,et al.  CRI-EMOA: A Pareto-Front Shape Invariant Evolutionary Multi-objective Algorithm , 2019, EMO.

[66]  Gary B. Lamont,et al.  Considerations in engineering parallel multiobjective evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[67]  Carlos A. Coello Coello,et al.  A GPU-Based Algorithm for a Faster Hypervolume Contribution Computation , 2015, EMO.

[68]  Wei Xin,et al.  Parallel quantum evolutionary algorithms with Client-Server model for multi-objective optimization on discrete problems , 2012, 2012 IEEE Congress on Evolutionary Computation.

[69]  Stefan Roth,et al.  Covariance Matrix Adaptation for Multi-objective Optimization , 2007, Evolutionary Computation.

[70]  Matjaz Depolli,et al.  Asynchronous Master-Slave Parallelization of Differential Evolution for Multi-Objective Optimization , 2013, Evolutionary Computation.

[71]  Julio Ortega Lopera,et al.  A Hybrid Scheme to Solve the Protein Structure Prediction Problem , 2010, IWPACBB.

[72]  Changhe Li,et al.  Handling Constrained Many-Objective Optimization Problems via Problem Transformation , 2020, IEEE Transactions on Cybernetics.

[73]  Shengxiang Yang,et al.  Shift-Based Density Estimation for Pareto-Based Algorithms in Many-Objective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[74]  Marc Schoenauer,et al.  Asynchronous Evolutionary Multi-Objective Algorithms with heterogeneous evaluation costs , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[75]  Markus Olhofer,et al.  Test Problems for Large-Scale Multiobjective and Many-Objective Optimization , 2017, IEEE Transactions on Cybernetics.

[76]  Carlos A. Coello Coello,et al.  A Review of Techniques for Handling Expensive Functions in Evolutionary Multi-Objective Optimization , 2010 .

[77]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[78]  El-Ghazali Talbi,et al.  A unified view of parallel multi-objective evolutionary algorithms , 2019, J. Parallel Distributed Comput..

[79]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[80]  Dario Izzo,et al.  The asynchronous island model and NSGA-II: study of a new migration operator and its performance , 2013, GECCO '13.

[81]  Xin Yao,et al.  Two_Arch2: An Improved Two-Archive Algorithm for Many-Objective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[82]  Carlos A. Coello Coello,et al.  Coevolutionary Operations for Large Scale Multi-objective Optimization , 2020, 2020 IEEE Congress on Evolutionary Computation (CEC).

[83]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[84]  Hisao Ishibuchi,et al.  Modified Distance Calculation in Generational Distance and Inverted Generational Distance , 2015, EMO.

[85]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[86]  Ye Tian,et al.  PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization [Educational Forum] , 2017, IEEE Computational Intelligence Magazine.

[87]  Scott Shenker,et al.  Spark: Cluster Computing with Working Sets , 2010, HotCloud.

[88]  Aurora Trinidad Ramirez Pozo,et al.  A GPU Implementation of MOEA/D-ACO for the Multiobjective Traveling Salesman Problem , 2014, 2014 Brazilian Conference on Intelligent Systems.

[89]  Jesús González,et al.  Multi-objective feature selection for EEG classification with multi-level parallelism on heterogeneous CPU-GPU clusters , 2018, GECCO.

[90]  Carlos A. Coello Coello,et al.  Solving Multiobjective Optimization Problems Using an Artificial Immune System , 2005, Genetic Programming and Evolvable Machines.

[91]  Enrique Alba,et al.  A cellular multi-objective genetic algorithm for optimal broadcasting strategy in metropolitan MANETs , 2007, Comput. Commun..

[92]  Gara Miranda,et al.  Parallel hyperheuristic: a self-adaptive island-based model for multi-objective optimization , 2008, GECCO '08.

[93]  Ian C. Parmee,et al.  Preliminary airframe design using co-evolutionary multiobjective genetic algorithms , 1999 .

[94]  Hiroyuki Sato,et al.  A Study for Parallelization of Multi-Objective Evolutionary Algorithm Based on Decomposition and Directed Mating , 2019 .

[95]  Carlos A. Coello Coello,et al.  Effective ranking + speciation = Many-objective optimization , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[96]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[97]  Dana Petcu,et al.  Adaptive Pareto Differential Evolution and Its Parallelization , 2003, PPAM.

[98]  Tomoyuki Hiroyasu,et al.  MOGADES: Multi-Objective Genetic Algorithm with Distributed Environment Scheme , 2005, Evolutionary Multiobjective Optimization.

[99]  Magdalene Marinaki,et al.  A Parallel Multi-Start NSGA II Algorithm for Multiobjective Energy Reduction Vehicle Routing Problem , 2015, EMO.

[100]  Dana Petcu,et al.  A Hierarchical Approach in Distributed Evolutionary Algorithms for Multiobjective Optimization , 2009, LSSC.

[101]  Junichi Suzuki,et al.  R2-IBEA: R2 indicator based evolutionary algorithm for multiobjective optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[102]  George Coulouris,et al.  Distributed systems - concepts and design , 1988 .

[103]  Kalyanmoy Deb,et al.  Pymoo: Multi-Objective Optimization in Python , 2020, IEEE Access.

[104]  Peter J. Fleming,et al.  Preference-inspired co-evolutionary algorithms using weight vectors , 2015, Eur. J. Oper. Res..

[105]  Tomoyuki Hiroyasu,et al.  DCMOGADES: distributed cooperation model of multi-objective genetic algorithm with distributed scheme , 2002 .

[106]  David W. Corne,et al.  No Free Lunch and Free Leftovers Theorems for Multiobjective Optimisation Problems , 2003, EMO.

[107]  Enrique Alba,et al.  Parallel Evolutionary Multiobjective Optimization , 2006, Parallel Evolutionary Computations.

[108]  Christian Grimme,et al.  Parallel predator–prey interaction for evolutionary multi-objective optimization , 2011, Natural Computing.

[109]  Christian Grimme,et al.  The Parallel Predator-Prey Model: A Step towards Practical Application , 2008, PPSN.

[110]  Julius Zilinskas,et al.  Parallel Multi-objective Memetic Algorithm for Competitive Facility Location , 2013, PPAM.

[111]  Carlos A. Coello Coello,et al.  Coevolutionary Multiobjective Evolutionary Algorithms: Survey of the State-of-the-Art , 2018, IEEE Trans. Evol. Comput..

[112]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[113]  Qingfu Zhang,et al.  MOEA/D-ACO: A Multiobjective Evolutionary Algorithm Using Decomposition and AntColony , 2013, IEEE Transactions on Cybernetics.

[114]  Francisco Luna,et al.  Parallel Multiobjective Optimization , 2005 .

[115]  Carlos A. Coello Coello,et al.  A hyper-heuristic of scalarizing functions , 2017, GECCO.

[116]  Enrique Alba,et al.  A Parallel Island Model for Hypervolume-Based Many-Objective Optimization , 2020, High-Performance Simulation-Based Optimization.

[117]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[118]  Bin Wu,et al.  Distributed Parellel MOEA/D on Spark , 2017, 2017 International Conference on Computing Intelligence and Information System (CIIS).

[119]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[120]  Gara Miranda,et al.  A Parallel Plugin-Based Framework for Multi-objective Optimization , 2009, DCAI.

[121]  Bilel Derbel,et al.  A fine-grained message passing MOEA/D , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[122]  Michael T. M. Emmerich,et al.  Test Problems Based on Lamé Superspheres , 2007, EMO.

[123]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[124]  Eduardo Segredo,et al.  On the Comparison of Parallel Island-Based Models for the Multiobjectivised Antenna Positioning Problem , 2011, KES.

[125]  Carlos A. Coello Coello,et al.  A Co-Evolutionary Scheme for Multi-Objective Evolutionary Algorithms Based on $\epsilon$ -Dominance , 2019, IEEE Access.

[126]  A. Rama Mohan Rao,et al.  Distributed evolutionary multi-objective mesh-partitioning algorithm for parallel finite element computations , 2009 .

[127]  Sushil J. Louis,et al.  Parallel implementation of niched Pareto genetic algorithm code for X-ray plasma spectroscopy , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[128]  Enrique Alba,et al.  A Multi-Objective Evolutionary Algorithm based on Parallel Coordinates , 2016, GECCO.

[129]  Enrique Alba,et al.  Parallel Multiobjective Evolutionary Algorithms , 2015, Handbook of Computational Intelligence.

[130]  Marc P. Armstrong,et al.  A Specialized Island Model and Its Application in Multiobjective Optimization , 2003, GECCO.

[131]  Tao Ma,et al.  A Survey of Multiobjective Evolutionary Algorithms Based on Decomposition: Variants, Challenges and Future Directions , 2020, IEEE Access.

[132]  Samarth Gupta,et al.  A scalable parallel implementation of evolutionary algorithms for multi-objective optimization on GPUs , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[133]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[134]  Weiwei Zhang,et al.  Cooperative Differential Evolution With Multiple Populations for Multiobjective Optimization , 2016, IEEE Transactions on Cybernetics.

[135]  Carlos A. Coello Coello,et al.  A non-cooperative game for faster convergence in cooperative coevolution for multi-objective optimization , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[136]  Wei-jie Yu,et al.  A parallel double-level multiobjective evolutionary algorithm for robust optimization , 2017, Appl. Soft Comput..

[137]  Eduardo Segredo,et al.  Parallel island-based multiobjectivised memetic algorithms for a 2D packing problem , 2011, GECCO '11.

[138]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[139]  Hiroyuki Sato,et al.  Directed mating in decomposition-based MOEA for constrained many-objective optimization , 2018, GECCO.

[140]  R. Lyndon While,et al.  A review of multiobjective test problems and a scalable test problem toolkit , 2006, IEEE Transactions on Evolutionary Computation.

[141]  Carlos A. Coello Coello,et al.  Applications of Parallel Platforms and Models in Evolutionary Multi-Objective Optimization , 2009 .

[142]  Carlos A. Coello Coello,et al.  A Parallel Multi-objective Memetic Algorithm Based on the IGD+ Indicator , 2016, PPSN.

[143]  Yanfang Liu,et al.  A parallelized multi-objective particle swarm optimization model to design soil sampling network , 2012, 2012 20th International Conference on Geoinformatics.