Objective Reduction in Many-Objective Optimization: Linear and Nonlinear Algorithms

The difficulties faced by existing multiobjective evolutionary algorithms (MOEAs) in handling many-objective problems relate to the inefficiency of selection operators, high computational cost, and difficulty in visualization of objective space. While many approaches aim to counter these difficulties by increasing the fidelity of the standard selection operators, the objective reduction approach attempts to eliminate objectives that are not essential to describe the Pareto-optimal front (POF). If the number of essential objectives is found to be two or three, the problem could be solved by the existing MOEAs. It implies that objective reduction could make an otherwise unsolvable (many-objective) problem solvable. Even when the essential objectives are four or more, the reduced representation of the problem will have favorable impact on the search efficiency, computational cost, and decision-making. Hence, development of generic and robust objective reduction approaches becomes important. This paper presents a principal component analysis and maximum variance unfolding based framework for linear and nonlinear objective reduction algorithms, respectively. The major contribution of this paper includes: 1) the enhancements in the core components of the framework for higher robustness in terms of applicability to a range of problems with disparate degree of redundancy; mechanisms to handle input data that poorly approximates the true POF; and dependence on fewer parameters to minimize the variability in performance; 2) proposition of an error measure to assess the quality of results; 3) sensitivity analysis of the proposed algorithms for the critical parameter involved, and the characteristics of the input data; and 4) study of the performance of the proposed algorithms vis-à-vis dominance relation preservation based algorithms, on a wide range of test problems (scaled up to 50 objectives) and two real-world problems.

[1]  Carlos M. Fonseca,et al.  An Improved Dimension-Sweep Algorithm for the Hypervolume Indicator , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[2]  Marco Laumanns,et al.  Scalable test problems for evolutionary multi-objective optimization , 2001 .

[3]  Mario Köppen,et al.  Substitute Distance Assignments in NSGA-II for Handling Many-objective Optimization Problems , 2007, EMO.

[4]  Eckart Zitzler,et al.  Evolutionary algorithms for multiobjective optimization: methods and applications , 1999 .

[5]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[6]  Alexander V. Lotov,et al.  Interactive Decision Maps , 2013 .

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  D KnowlesJoshua,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000 .

[9]  Tapabrata Ray,et al.  A Pareto Corner Search Evolutionary Algorithm and Dimensionality Reduction in Many-Objective Optimization Problems , 2011, IEEE Transactions on Evolutionary Computation.

[10]  Peter J. Fleming,et al.  Evolutionary many-objective optimisation: an exploratory analysis , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[11]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[12]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[13]  Kiyoshi Tanaka,et al.  Controlling Dominance Area of Solutions and Its Impact on the Performance of MOEAs , 2007, EMO.

[14]  Carlos A. Coello Coello,et al.  Online Objective Reduction to Deal with Many-Objective Problems , 2009, EMO.

[15]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[16]  Marco Laumanns,et al.  Scalable Test Problems for Evolutionary Multiobjective Optimization , 2005, Evolutionary Multiobjective Optimization.

[17]  Rolf Drechsler,et al.  Multi-objective Optimisation Based on Relation Favour , 2001, EMO.

[18]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, CVPR.

[19]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[20]  LiHui,et al.  Multiobjective optimization problems with complicated Pareto sets, MOEA/D and NSGA-II , 2009 .

[21]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[22]  Hisao Ishibuchi,et al.  Effects of the Existence of Highly Correlated Objectives on the Behavior of MOEA/D , 2011, EMO.

[23]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[24]  Nicola Beume,et al.  Pareto-, Aggregation-, and Indicator-Based Methods in Many-Objective Optimization , 2007, EMO.

[25]  David W. Corne,et al.  Quantifying the Effects of Objective Space Dimension in Evolutionary Multiobjective Optimization , 2007, EMO.

[26]  Marco Laumanns,et al.  Combining Convergence and Diversity in Evolutionary Multiobjective Optimization , 2002, Evolutionary Computation.

[27]  Kilian Q. Weinberger,et al.  Spectral Methods for Dimensionality Reduction , 2006, Semi-Supervised Learning.

[28]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[29]  Carlos A. Coello Coello,et al.  Some techniques to deal with many-objective problems , 2009, GECCO '09.

[30]  B. Borchers CSDP, A C library for semidefinite programming , 1999 .

[31]  Qingfu Zhang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 RM-MEDA: A Regularity Model-Based Multiobjective Estimation of , 2022 .

[32]  Hisao Ishibuchi,et al.  Incorporation of Scalarizing Fitness Functions into Evolutionary Multiobjective Optimization Algorithms , 2006, PPSN.

[33]  Kalyanmoy Deb,et al.  Evaluating the -Domination Based Multi-Objective Evolutionary Algorithm for a Quick Computation of Pareto-Optimal Solutions , 2005, Evolutionary Computation.

[34]  R. Lyndon While,et al.  A faster algorithm for calculating hypervolume , 2006, IEEE Transactions on Evolutionary Computation.

[35]  Kalyanmoy Deb,et al.  Integrating User Preferences into Evolutionary Multi-Objective Optimization , 2005 .

[36]  Yong Wang,et al.  A regularity model-based multiobjective estimation of distribution algorithm with reducing redundant cluster operator , 2012, Appl. Soft Comput..

[37]  Hisao Ishibuchi,et al.  Iterative approach to indicator-based multiobjective optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[38]  Daisuke Sasaki,et al.  Visualization and Data Mining of Pareto Solutions Using Self-Organizing Map , 2003, EMO.

[39]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[40]  Kaisa Miettinen,et al.  A Preference Based Interactive Evolutionary Algorithm for Multi-objective Optimization: PIE , 2011, EMO.

[41]  Joseph J. Talavage,et al.  A Tradeoff Cut Approach to Multiple Objective Optimization , 1980, Oper. Res..

[42]  Xin Yao,et al.  Performance Scaling of Multi-objective Evolutionary Algorithms , 2003, EMO.

[43]  Carlos A. Coello Coello,et al.  Objective reduction using a feature selection technique , 2008, GECCO '08.

[44]  Hisao Ishibuchi,et al.  Evolutionary many-objective optimization: A short review , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[45]  Alice M. Agogino,et al.  Theory of design: An optimization perspective , 1990 .

[46]  Qingfu Zhang,et al.  Framework for Many-Objective Test Problems with Both Simple and Complicated Pareto-Set Shapes , 2011, EMO.

[47]  Jonathon Shlens,et al.  A Tutorial on Principal Component Analysis , 2014, ArXiv.

[48]  Kalyanmoy Deb,et al.  Non-linear Dimensionality Reduction Procedures for Certain Large-Dimensional Multi-objective Optimization Problems: Employing Correntropy and a Novel Maximum Variance Unfolding , 2007, EMO.

[49]  David W. Corne,et al.  Techniques for highly multiobjective optimisation: some nondominated points are better than others , 2007, GECCO '07.

[50]  Nicola Beume,et al.  S-Metric Calculation by Considering Dominated Hypervolume as Klee's Measure Problem , 2009, Evolutionary Computation.

[51]  R. Lyndon While,et al.  A review of multiobjective test problems and a scalable test problem toolkit , 2006, IEEE Transactions on Evolutionary Computation.

[52]  Eckart Zitzler,et al.  Are All Objectives Necessary? On Dimensionality Reduction in Evolutionary Multiobjective Optimization , 2006, PPSN.

[53]  Eckart Zitzler,et al.  Objective Reduction in Evolutionary Multiobjective Optimization: Theory and Applications , 2009, Evolutionary Computation.

[54]  Evan J. Hughes,et al.  Evolutionary many-objective optimisation: many once or one many? , 2005, 2005 IEEE Congress on Evolutionary Computation.

[55]  Dimo Brockhoff,et al.  Many objective optimization and hypervolume based search , 2009 .

[56]  Peter J. Fleming,et al.  Many-Objective Optimization: An Engineering Design Perspective , 2005, EMO.

[57]  Jouni Lampinen,et al.  Ranking-Dominance and Many-Objective Optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[58]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.