Does Preference Always Help? A Holistic Study on Preference-Based Evolutionary Multiobjective Optimization Using Reference Points

The ultimate goal of multiobjective optimization is to help a decision maker (DM) identify solution(s) of interest (SOI) achieving satisfactory tradeoffs among multiple conflicting criteria. This can be realized by leveraging DM’s preference information in evolutionary multiobjective optimization (EMO). No consensus has been reached on the effectiveness brought by incorporating preference in EMO (either a priori or interactively) versus a posteriori decision making after a complete run of an EMO algorithm. Bearing this consideration in mind, this article: 1) provides a pragmatic overview of the existing developments of preference-based EMO (PBEMO) and 2) conducts a series of experiments to investigate the effectiveness brought by preference incorporation in EMO for approximating various SOI. In particular, the DM’s preference information is elicited as a reference point, which represents her/his aspirations for different objectives. The experimental results demonstrate that preference incorporation in EMO does not always lead to a desirable approximation of SOI if the DM’s preference information is not well utilized, nor does the DM elicit invalid preference information, which is not uncommon when encountering a black-box system. To a certain extent, this issue can be remedied through an interactive preference elicitation. Last but not the least, we find that a PBEMO algorithm is able to be generalized to approximate the whole PF given an appropriate setup of preference information.

[1]  Markus Olhofer,et al.  A mini-review on preference modeling and articulation in multi-objective optimization: current status and challenges , 2017, Complex & Intelligent Systems.

[2]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[3]  Marouane Kessentini,et al.  Preference Incorporation in Evolutionary Multiobjective Optimization , 2015 .

[4]  Geyong Min,et al.  A Formal Model for Multi-objective Optimisation of Network Function Virtualisation Placement , 2019, EMO.

[5]  Jian-Bo Yang,et al.  PROJECT Method for Multiobjective Optimization Based on Gradient Projection and Reference Points , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[6]  Xiaoyan Sun,et al.  Many-objective evolutionary optimization based on reference points , 2017, Appl. Soft Comput..

[7]  Lothar Thiele,et al.  A Preference-Based Evolutionary Algorithm for Multi-Objective Optimization , 2009, Evolutionary Computation.

[8]  Bernhard Sendhoff,et al.  Preference representation using Gaussian functions on a hyperplane in evolutionary multi-objective optimization , 2016, Soft Comput..

[9]  Dirk Thierens,et al.  The balance between proximity and diversity in multiobjective evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[10]  Sam Kwong,et al.  AN indicator-based selection multi-objective evolutionary algorithm with preference for multi-class ensemble , 2014, 2014 International Conference on Machine Learning and Cybernetics.

[11]  Qingfu Zhang,et al.  Stable Matching-Based Selection in Evolutionary Multiobjective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[12]  Jing Sun,et al.  Interval Multiobjective Optimization With Memetic Algorithms , 2020, IEEE Transactions on Cybernetics.

[13]  J. Branke,et al.  Guidance in evolutionary multi-objective optimization , 2001 .

[14]  Xiaoyan Sun,et al.  A New Surrogate-Assisted Interactive Genetic Algorithm With Weighted Semisupervised Learning , 2013, IEEE Transactions on Cybernetics.

[15]  Xin Yao,et al.  Integration of Preferences in Decomposition Multiobjective Optimization , 2017, IEEE Transactions on Cybernetics.

[16]  Xiaoyan Sun,et al.  Set-based many-objective optimization guided by a preferred region , 2017, Neurocomputing.

[17]  Qingfu Zhang,et al.  Adaptive Operator Selection With Bandits for a Multiobjective Evolutionary Algorithm Based on Decomposition , 2014, IEEE Transactions on Evolutionary Computation.

[18]  Qingfu Zhang,et al.  Interactive MOEA/D for multi-objective decision making , 2011, GECCO '11.

[19]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[20]  Ricardo H. C. Takahashi,et al.  INSPM: An interactive evolutionary multi-objective algorithm with preference model , 2014, Inf. Sci..

[21]  Shengxiang Yang,et al.  A knee-point-based evolutionary algorithm using weighted subpopulation for many-objective optimization , 2019, Swarm Evol. Comput..

[22]  Jonathan E. Fieldsend,et al.  Life on the Edge: Characterising the Edges of Mutually Non-dominating Sets , 2014, Evolutionary Computation.

[23]  Jyrki Wallenius,et al.  Interactive evolutionary multi-objective optimization for quasi-concave preference functions , 2010, Eur. J. Oper. Res..

[24]  Kim-Fung Man,et al.  Learning paradigm based on jumping genes: A general framework for enhancing exploration in evolutionary multiobjective optimization , 2013, Inf. Sci..

[25]  Kaisa Miettinen,et al.  Interactive multiobjective optimization system WWW-NIMBUS on the Internet , 2000, Comput. Oper. Res..

[26]  Hisao Ishibuchi,et al.  Pareto Fronts of Many-Objective Degenerate Test Problems , 2016, IEEE Transactions on Evolutionary Computation.

[27]  Peter J. Fleming,et al.  Multiobjective optimization and multiple constraint handling with evolutionary algorithms. I. A unified formulation , 1998, IEEE Trans. Syst. Man Cybern. Part A.

[28]  Kaisa Miettinen,et al.  NAUTILUS method: An interactive technique in multiobjective optimization based on the nadir point , 2010, Eur. J. Oper. Res..

[29]  Bernhard Sendhoff,et al.  Incorporation Of Fuzzy Preferences Into Evolutionary Multiobjective Optimization , 2002, GECCO.

[30]  Qingfu Zhang,et al.  Two-Level Stable Matching-Based Selection in MOEA/D , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[31]  Ian C. Parmee,et al.  Preferences and their application in evolutionary multiobjective optimization , 2002, IEEE Trans. Evol. Comput..

[32]  Sam Kwong,et al.  Multi-objective differential evolution with self-navigation , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[33]  Xiaodong Li,et al.  Reference Point-Based Particle Swarm Optimization Using a Steady-State Approach , 2008, SEAL.

[34]  Kalyanmoy Deb,et al.  Interactive evolutionary multi-objective optimization and decision-making using reference direction method , 2007, GECCO '07.

[35]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[36]  Kalyanmoy Deb,et al.  A dual-population paradigm for evolutionary multiobjective optimization , 2015, Inf. Sci..

[37]  Kalyanmoy Deb,et al.  An Interactive Evolutionary Multi-objective Optimization Method Based on Polyhedral Cones , 2010, LION.

[38]  Hisao Ishibuchi,et al.  Interactive Multiobjective Optimization: A Review of the State-of-the-Art , 2018, IEEE Access.

[39]  Hisao Ishibuchi,et al.  Hypervolume Subset Selection for Triangular and Inverted Triangular Pareto Fronts of Three-Objective Problems , 2017, FOGA '17.

[40]  S. Greco,et al.  Axiomatization of utility, outranking and decision-rule preference models for multiple-criteria classification problems under partial inconsistency with the dominance principle , 2002 .

[41]  Khaled Ghédira,et al.  The r-Dominance: A New Dominance Relation for Interactive Evolutionary Multicriteria Decision Making , 2010, IEEE Transactions on Evolutionary Computation.

[42]  Xiaodong Li,et al.  Integrating user preferences and decomposition methods for many-objective optimization , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[43]  Kalyanmoy Deb,et al.  An Interactive Evolutionary Multiobjective Optimization Method Based on Progressively Approximated Value Functions , 2010, IEEE Transactions on Evolutionary Computation.

[44]  Jie Chen,et al.  A Tradeoff-Based Interactive Multi-Objective Optimization Method Driven by Evolutionary Algorithms , 2017, J. Adv. Comput. Intell. Intell. Informatics.

[45]  Hisao Ishibuchi,et al.  Behavior of EMO algorithms on many-objective optimization problems with correlated objectives , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[46]  Zhen Zhang,et al.  Novel Interactive Preference-Based Multiobjective Evolutionary Optimization for Bolt Supporting Networks , 2020, IEEE Transactions on Evolutionary Computation.

[47]  Sam Kwong,et al.  Class-specific soft voting based multiple extreme learning machines ensemble , 2015, Neurocomputing.

[48]  Fang Liu,et al.  MOEA/D with biased weight adjustment inspired by user preference and its application on multi-objective reservoir flood control problem , 2016, Soft Comput..

[49]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[50]  Hisao Ishibuchi,et al.  Behavior of Multiobjective Evolutionary Algorithms on Many-Objective Knapsack Problems , 2015, IEEE Transactions on Evolutionary Computation.

[51]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Tobias Friedrich,et al.  Approximation quality of the hypervolume indicator , 2013, Artif. Intell..

[53]  Xin Yao,et al.  Two-Archive Evolutionary Algorithm for Constrained Multiobjective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[54]  Murat Köksalan,et al.  An Interactive Territory Defining Evolutionary Algorithm: iTDEA , 2010, IEEE Transactions on Evolutionary Computation.

[55]  Carlos A. Coello Coello,et al.  Handling preferences in evolutionary multiobjective optimization: a survey , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[56]  Peter J. Fleming,et al.  Towards Understanding the Cost of Adaptation in Decomposition-Based Optimization Algorithms , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[57]  Xiaojun Zeng,et al.  An ensemble framework for assessing solutions of interval programming problems , 2018, Inf. Sci..

[58]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[59]  Garrison W. Greenwood,et al.  Fitness Functions for Multiple Objective Optimization Problems: Combining Preferences with Pareto Rankings , 1996, FOGA.

[60]  Roberto Battiti,et al.  Brain-Computer Evolutionary Multiobjective Optimization: A Genetic Algorithm Adapting to the Decision Maker , 2010, IEEE Trans. Evol. Comput..

[61]  Sam Kwong,et al.  EVOLVING EXTREME LEARNING MACHINE PARADIGM WITH ADAPTIVE OPERATOR SELECTION AND PARAMETER CONTROL , 2013 .

[62]  Lothar Thiele,et al.  The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration , 2007, EMO.

[63]  Kalyanmoy Deb,et al.  Multi-objective evolutionary algorithms: introducing bias among Pareto-optimal solutions , 2003 .

[64]  K. Deb,et al.  Understanding knee points in bicriteria problems and their implications as preferred solution principles , 2011 .

[65]  Qingfu Zhang,et al.  Interrelationship-Based Selection for Decomposition Multiobjective Optimization , 2015, IEEE Transactions on Cybernetics.

[66]  Jinhua Zheng,et al.  Decomposing the user-preference in multiobjective optimization , 2016, Soft Comput..

[67]  Sam Kwong,et al.  A general framework for evolutionary multiobjective optimization via manifold learning , 2014, Neurocomputing.

[68]  Tapabrata Ray,et al.  Distance-Based Subset Selection for Benchmarking in Evolutionary Multi/Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[69]  Ke Li,et al.  Progressive Preference Learning: Proof-of-Principle Results in MOEA/D , 2019, EMO.

[70]  Salvatore Greco,et al.  Interactive Multiobjective Mixed-Integer Optimization Using Dominance-Based Rough Set Approach , 2011, EMO.

[71]  Dun-Wei Gong,et al.  Evolutionary algorithms with preference polyhedron for interval multi-objective optimization problems , 2013, Inf. Sci..

[72]  Xin Yao,et al.  Many-Objective Evolutionary Algorithms , 2015, ACM Comput. Surv..

[73]  Qingfu Zhang,et al.  Efficient Nondomination Level Update Method for Steady-State Evolutionary Multiobjective Optimization , 2017, IEEE Transactions on Cybernetics.

[74]  Qingfu Zhang,et al.  Learning to Decompose: A Paradigm for Decomposition-Based Multiobjective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[75]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[76]  Xin Yao,et al.  Interactive Decomposition Multiobjective Optimization Via Progressively Learned Value Functions , 2018, IEEE Transactions on Fuzzy Systems.

[77]  Murat Köksalan,et al.  An Interactive Evolutionary Metaheuristic for Multiobjective Combinatorial Optimization , 2003, Manag. Sci..

[78]  Qingfu Zhang,et al.  Adaptive weights generation for decomposition-based multi-objective optimization using Gaussian process regression , 2017, GECCO.

[79]  Cong Zhou,et al.  A novel algorithm for non-dominated hypervolume-based multiobjective optimization , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[80]  Kin Keung Lai,et al.  Mean-Variance-Skewness-Kurtosis-based Portfolio Optimization , 2006, First International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06).

[81]  Álvaro Fialho,et al.  Multi-Objective Differential Evolution with Adaptive Control of Parameters and Operators , 2011, LION.

[82]  Tapabrata Ray,et al.  Bridging the Gap: Many-Objective Optimization and Informed Decision-Making , 2017, IEEE Transactions on Evolutionary Computation.

[83]  Qingfu Zhang,et al.  An Evolutionary Many-Objective Optimization Algorithm Based on Dominance and Decomposition , 2015, IEEE Transactions on Evolutionary Computation.

[84]  Qingfu Zhang,et al.  Matching-Based Selection With Incomplete Lists for Decomposition Multiobjective Optimization , 2016, IEEE Transactions on Evolutionary Computation.

[85]  Hisao Ishibuchi,et al.  On Scalable Multiobjective Test Problems With Hardly Dominated Boundaries , 2019, IEEE Transactions on Evolutionary Computation.

[86]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[87]  Jürgen Branke,et al.  Learning Value Functions in Interactive Evolutionary Multiobjective Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[88]  Xin Yao,et al.  Empirical Investigations of Reference Point Based Methods When Facing a Massively Large Number of Objectives: First Results , 2017, EMO.

[89]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[90]  Kalyanmoy Deb,et al.  Light beam search based multi-objective optimization using evolutionary algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[91]  Heike Trautmann,et al.  Preference Articulation by Means of the R2 Indicator , 2013, EMO.

[92]  Qingfu Zhang,et al.  Evolutionary Many-Objective Optimization Based on Adversarial Decomposition , 2017, IEEE Transactions on Cybernetics.

[93]  Hisao Ishibuchi,et al.  Performance of Decomposition-Based Many-Objective Algorithms Strongly Depends on Pareto Front Shapes , 2017, IEEE Transactions on Evolutionary Computation.

[94]  Xiaodong Li,et al.  Integrating user preferences with particle swarms for multi-objective optimization , 2008, GECCO '08.

[95]  Frank Neumann,et al.  Weighted preferences in evolutionary multi-objective optimization , 2013, Int. J. Mach. Learn. Cybern..

[96]  Qingwei Chen,et al.  A multi-objective optimization evolutionary algorithm incorporating preference information based on fuzzy logic , 2010, Comput. Optim. Appl..

[97]  Cong Zhou,et al.  An Improved Differential Evolution for Multi-objective Optimization , 2009, 2009 WRI World Congress on Computer Science and Information Engineering.

[98]  Kaisa Miettinen,et al.  A Preference Based Interactive Evolutionary Algorithm for Multi-objective Optimization: PIE , 2011, EMO.

[99]  Carlos A. Coello Coello,et al.  g-dominance: Reference point based dominance for multiobjective metaheuristics , 2009, Eur. J. Oper. Res..

[100]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[101]  Jian-Bo Yang,et al.  Normal vector identification and interactive tradeoff analysis using minimax formulation in multiobjective optimization , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[102]  Heike Trautmann,et al.  Integration of Preferences in Hypervolume-Based Multiobjective Evolutionary Algorithms by Means of Desirability Functions , 2010, IEEE Transactions on Evolutionary Computation.

[103]  Murat Köksalan,et al.  A Territory Defining Multiobjective Evolutionary Algorithms and Preference Incorporation , 2010, IEEE Transactions on Evolutionary Computation.

[104]  K. Metaxiotis A Mean–Variance–Skewness Portfolio Optimization Model , 2019 .

[105]  Kalyanmoy Deb,et al.  Integrating User Preferences into Evolutionary Multi-Objective Optimization , 2005 .

[106]  Lily Rachmawati,et al.  Preference Incorporation in Multi-objective Evolutionary Algorithms: A Survey , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[107]  Kalyanmoy Deb,et al.  A review of hybrid evolutionary multiple criteria decision making methods , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[108]  Xin Yao,et al.  R-Metric: Evaluating the Performance of Preference-Based Evolutionary Multiobjective Optimization Using Reference Points , 2018, IEEE Transactions on Evolutionary Computation.

[109]  Heike Trautmann,et al.  R2-EMOA: Focused Multiobjective Search Using R2-Indicator-Based Selection , 2013, LION.

[110]  Xin Yao,et al.  FEMOSAA: Feature Guided and Knee Driven Multi-Objective Optimization for Self-Adaptive Software at Runtime , 2016, ACM Trans. Softw. Eng. Methodol..

[111]  Jian-Bo Yang,et al.  Gradient projection and local region search for multiobjective optimisation , 1999, Eur. J. Oper. Res..

[112]  Xin Yao,et al.  Dynamic Multiobjectives Optimization With a Changing Number of Objectives , 2016, IEEE Transactions on Evolutionary Computation.

[113]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.

[114]  Sam Kwong,et al.  A weighted voting method using minimum square error based on Extreme Learning Machine , 2012, 2012 International Conference on Machine Learning and Cybernetics.