Pymoo: Multi-Objective Optimization in Python

Python has become the programming language of choice for research and industry projects related to data science, machine learning, and deep learning. Since optimization is an inherent part of these research fields, more optimization related frameworks have arisen in the past few years. Only a few of them support optimization of multiple conflicting objectives at a time, but do not provide comprehensive tools for a complete multi-objective optimization task. To address this issue, we have developed pymoo, a multi-objective optimization framework in Python. We provide a guide to getting started with our framework by demonstrating the implementation of an exemplary constrained multi-objective optimization scenario. Moreover, we give a high-level overview of the architecture of pymoo to show its capabilities followed by an explanation of each module and its corresponding sub-modules. The implementations in our framework are customizable and algorithms can be modified/extended by supplying custom operators. Moreover, a variety of single, multi- and many-objective test problems are provided and gradients can be retrieved by automatic differentiation out of the box. Also, pymoo addresses practical needs, such as the parallelization of function evaluations, methods to visualize low and high-dimensional spaces, and tools for multi-criteria decision making. For more information about pymoo, readers are encouraged to visit: https://pymoo.org.

[1]  Sanaz Mostaghim,et al.  Heatmap Visualization of Population Based Multi Objective Algorithms , 2007, EMO.

[2]  Kalyanmoy Deb,et al.  A Running Performance Metric and Termination Criterion for Evaluating Evolutionary Multi- and Many-objective Optimization Algorithms , 2020, 2020 IEEE Congress on Evolutionary Computation (CEC).

[3]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[4]  Ashwin Pajankar.  Python Unit Test Automation: Practical Techniques for Python Developers and Testers , 2017 .

[5]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[6]  Georges G. Grinstein,et al.  Dimensional anchors: a graphic primitive for multidimensional multivariate information visualizations , 1999, NPIVM '99.

[7]  Andrzej P. Wierzbicki,et al.  The Use of Reference Objectives in Multiobjective Optimization , 1979 .

[8]  G. vanRossum.  Python reference manual , 1995 .

[9]  Kalyanmoy Deb,et al.  A flexible optimization procedure for mechanical component design based on genetic adaptive search , 1998 .

[10]  D. Corne,et al.  On Metrics for Comparing Non Dominated Sets , 2001 .

[11]  Dario Izzo,et al.  FOR MASSIVELY PARALLEL OPTIMIZATION IN ASTRODYNAMICS ( THE CASE OF INTERPLANETARY TRAJECTORY OPTIMIZATION ) , 2012 .

[12]  Kalyanmoy Deb,et al.  Self-adaptive simulated binary crossover for real-parameter optimization , 2007, GECCO '07.

[13]  Kalyanmoy Deb,et al.  A Unified Evolutionary Optimization Procedure for Single, Multiple, and Many Objectives , 2016, IEEE Transactions on Evolutionary Computation.

[14]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[15]  Guy L. Steele,et al.  The Java Language Specification , 1996 .

[16]  Niall M. Fraser,et al.  The modified star graph and the petal diagram: two new visual aids for discrete alternative multicriteria decision making , 1998 .

[17]  Lothar Thiele,et al.  The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration , 2007, EMO.

[18]  Uwe Naumann,et al.  Automatic Differentiation: Applications, Theory, and Implementations (Lecture Notes in Computational Science and Engineering) , 2006 .

[19]  Hisao Ishibuchi,et al.  Modified Distance Calculation in Generational Distance and Inverted Generational Distance , 2015, EMO.

[20]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[21]  E. Wegman.  Hyperdimensional Data Analysis Using Parallel Coordinates , 1990 .

[22]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[23]  Kalyanmoy Deb,et al.  A population-based fast algorithm for a billion-dimensional resource allocation problem with integer variables , 2017, Eur. J. Oper. Res..

[24]  Stefan Behnel,et al.  Cython: The Best of Both Worlds , 2011, Computing in Science & Engineering.

[25]  Kalyanmoy Deb,et al.  Optimization for Engineering Design: Algorithms and Examples , 2004 .

[26]  Milan Zeleny,et al.  Gestalt system of holistic graphics: New management support view of MCDM , 1991, Comput. Oper. Res..

[27]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[28]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[29]  Lothar Thiele,et al.  Multiobjective Optimization Using Evolutionary Algorithms - A Comparative Case Study , 1998, PPSN.

[30]  James C. Bean,et al.  Genetic Algorithms and Random Keys for Sequencing and Optimization , 1994, INFORMS J. Comput..

[31]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[32]  Richard Curran,et al.  Optimization of noise abatement aircraft terminal routes using a multi-objective evolutionary algorithm based on decomposition , 2018 .

[33]  Bernabé Dorronsoro,et al.  A Survey of Decomposition Methods for Multi-objective Optimization , 2014, Recent Advances on Hybrid Approaches for Designing Intelligent Systems.

[34]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[35]  Kalyanmoy Deb,et al.  An Interactive Evolutionary Multiobjective Optimization Method Based on Progressively Approximated Value Functions , 2010, IEEE Transactions on Evolutionary Computation.

[36]  A. Wierzbicki.  A Mathematical Basis for Satisficing Decision Making , 1982 .

[37]  Lily Rachmawati,et al.  Multiobjective Evolutionary Algorithm With Controllable Focus on the Knees of the Pareto Front , 2009, IEEE Transactions on Evolutionary Computation.

[38]  Kalyanmoy Deb,et al.  An Optimality Theory-Based Proximity Measure for Set-Based Multiobjective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[39]  Kalyanmoy Deb,et al.  PaletteViz: A Visualization Method for Functional Understanding of High-Dimensional Pareto-Optimal Data-Sets to Aid Multi-Criteria Decision Making , 2020, IEEE Computational Intelligence Magazine.

[40]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[41]  Ye Tian,et al.  PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization [Educational Forum] , 2017, IEEE Computational Intelligence Magazine.

[42]  Kalyanmoy Deb,et al.  Investigating the Normalization Procedure of NSGA-III , 2019, EMO.

[43]  José Francisco Aldana Montes,et al.  jMetalCpp: optimizing molecular docking problems with a C++ metaheuristic framework , 2014, Bioinform..

[44]  Beat Kleiner,et al.  10 Graphical techniques for multivariate data and for clustering , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[45]  Sriparna Saha,et al.  Fast implementation of steady-state NSGA-II , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[46]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[47]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[48]  Kalyanmoy Deb,et al.  Generating Well-Spaced Points on a Unit Simplex for Evolutionary Many-Objective Optimization , 2020, IEEE Transactions on Evolutionary Computation.

[49]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[50]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[51]  Dario Izzo,et al.  A Global Optimisation Toolbox for Massively Parallel Engineering Optimisation , 2010, ArXiv.

[52]  K. Deb,et al.  A bi-objective constrained optimization algorithm using a hybrid evolutionary and penalty function approach , 2013 .

[53]  Guy L. Steele,et al.  The Java Language Specification, Java SE 8 Edition , 2013 .

[54]  Javier Del Ser,et al.  jMetalPy: a Python Framework for Multi-Objective Optimization with Metaheuristics , 2019, Swarm Evol. Comput..

[55]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[56]  Eser Kandogan.  Star Coordinates: A Multi-dimensional Visualization Technique with Uniform Treatment of Dimensions , 2000 .

[57]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[58]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[59]  Kalyanmoy Deb,et al.  Reference Point Based NSGA-III for Preferred Solutions , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[60]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.

[61]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[62]  Kalyanmoy Deb,et al.  Analysing mutation schemes for real-parameter genetic algorithms , 2014, Int. J. Artif. Intell. Soft Comput..