Turning High-Dimensional Optimization Into Computationally Expensive Optimization

Divide-and-conquer (DC) is conceptually well suited to deal with high-dimensional optimization problems by decomposing the original problem into multiple low-dimensional subproblems, and tackling them separately. Nevertheless, the dimensionality mismatch between the original problem and subproblems makes it nontrivial to precisely assess the quality of a candidate solution to a subproblem, which has been a major hurdle for applying the idea of DC to nonseparable high-dimensional optimization problems. In this paper, we suggest that searching a good solution to a subproblem can be viewed as a computationally expensive problem and can be addressed with the aid of meta-models. As a result, a novel approach, namely self-evaluation evolution (SEE) is proposed. Empirical studies have shown the advantages of SEE over four representative compared algorithms increase with the problem size on the CEC2010 large scale global optimization benchmark. The weakness of SEE is also analyzed in the empirical studies.

[1]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[2]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[3]  Kalyanmoy Deb,et al.  RapidAccurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms , 1993, ICGA.

[4]  Xin Yao,et al.  Self-adaptive differential evolution with neighborhood search , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[5]  S. Crawford,et al.  Volume 1 , 2012, Journal of Diabetes Investigation.

[6]  R. Paul Wiegand,et al.  Spatial Embedding and Loss of Gradient in Cooperative Coevolutionary Algorithms , 2004, PPSN.

[7]  Larry Bull,et al.  Toward the Coevolution of Novel Vertical-Axis Wind Turbines , 2012, IEEE Transactions on Evolutionary Computation.

[8]  R. Paul Wiegand,et al.  An empirical analysis of collaboration methods in cooperative coevolutionary algorithms , 2001 .

[9]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[10]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[11]  Kalyanmoy Deb,et al.  Breaking the Billion-Variable Barrier in Real-World Optimization Using a Customized Evolutionary Algorithm , 2016, GECCO.

[12]  Hai Yang,et al.  ACM Transactions on Intelligent Systems and Technology - Special Section on Urban Computing , 2014 .

[13]  Liviu Panait,et al.  Theoretical Convergence Guarantees for Cooperative Coevolutionary Algorithms , 2010, Evolutionary Computation.

[14]  Anne Auger,et al.  Log-Linear Convergence and Optimal Bounds for the (1+1)-ES , 2007, Artificial Evolution.

[15]  Ata Kabán,et al.  Toward Large-Scale Continuous EDA: A Random Matrix Theory Perspective , 2013, Evolutionary Computation.

[16]  Jeffrey K. Bassett,et al.  An Analysis of Cooperative Coevolutionary Algorithms A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at George Mason University , 2003 .

[17]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[18]  Xiaodong Li,et al.  Cooperatively Coevolving Particle Swarms for Large Scale Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[19]  Yang Yu,et al.  Scaling Simultaneous Optimistic Optimization for High-Dimensional Non-Convex Functions with Low Effective Dimensions , 2016, AAAI.

[20]  Anne Auger,et al.  Evolution Strategies , 2018, Handbook of Computational Intelligence.

[21]  Bart De Schutter,et al.  30th AAAI Conference on Artificial Intelligence, AAAI 2016 , 2016, AAAI 2016.

[22]  Pedro M. Domingos,et al.  Recursive Decomposition for Nonconvex Optimization , 2015, ArXiv.

[23]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[24]  Daniel Molina Cabrera Evolutionary algorithms for large-scale global optimisation: a snapshot, trends and challenges , 2016, Progress in Artificial Intelligence.

[25]  Alex A. Freitas,et al.  Evolutionary Computation , 2002 .

[26]  Francisco Herrera,et al.  MA-SW-Chains: Memetic algorithm based on local search chains for large scale continuous global optimization , 2010, IEEE Congress on Evolutionary Computation.

[27]  Zhenyu Yang,et al.  Large-Scale Global Optimization Using Cooperative Coevolution with Variable Interaction Learning , 2010, PPSN.

[28]  B. C. Brookes,et al.  Information Sciences , 2020, Cognitive Skills You Need for the 21st Century.

[29]  Lukás Sekanina,et al.  Evolutionary Approach to Approximate Digital Circuits Design , 2015, IEEE Transactions on Evolutionary Computation.

[30]  Nando de Freitas,et al.  Bayesian optimization in high dimensions via random embeddings , 2013, IJCAI 2013.

[31]  Jonathan J. Hull,et al.  A Database for Handwritten Text Recognition Research , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Ken Lang,et al.  NewsWeeder: Learning to Filter Netnews , 1995, ICML.

[33]  Xin Yao,et al.  Fast Evolution Strategies , 1997, Evolutionary Programming.

[34]  Qingfu Zhang,et al.  An Estimation of Distribution Algorithm With Cheap and Expensive Local Search Methods , 2015, IEEE Transactions on Evolutionary Computation.

[35]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[36]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[37]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[38]  YaoXin,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008 .

[39]  X. Yao,et al.  Scaling up fast evolutionary programming with cooperative coevolution , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[40]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[41]  Sean Luke,et al.  Archive-based cooperative coevolutionary algorithms , 2006, GECCO '06.

[42]  Mark Harman,et al.  Ieee Transactions on Evolutionary Computation 1 , 2022 .

[43]  Thomas Jansen,et al.  Exploring the Explorative Advantage of the Cooperative Coevolutionary (1+1) EA , 2003, GECCO.

[44]  Petros Koumoutsakos,et al.  Learning probability distributions in continuous evolutionary algorithms – a comparative review , 2004, Natural Computing.

[45]  Yoel Tenne,et al.  A framework for memetic optimization using variable global and local surrogate models , 2009, Soft Comput..

[46]  Xiaodong Li,et al.  Cooperative Coevolution With Route Distance Grouping for Large-Scale Capacitated Arc Routing Problems , 2014, IEEE Transactions on Evolutionary Computation.

[47]  David E. Goldberg,et al.  Dependency Structure Matrix, Genetic Algorithms, and Effective Recombination , 2009, Evolutionary Computation.

[48]  Marjan Mernik,et al.  Exploration and exploitation in evolutionary algorithms: A survey , 2013, CSUR.

[49]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[50]  Yaochu Jin,et al.  A Competitive Swarm Optimizer for Large Scale Optimization , 2015, IEEE Transactions on Cybernetics.

[51]  D.E. Goldberg,et al.  A genetic algorithm using linkage identification by nonlinearity check , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[52]  Richard A. Watson,et al.  Analysis of recombinative algorithms on a non-separable building-block problem , 2000, FOGA.

[53]  Gianni Di Pillo,et al.  Support vector machines for surrogate modeling of electronic circuits , 2013, Neural Computing and Applications.

[54]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[55]  幸治 下山 2014 IEEE World Congress on Computational Intelligenceに参加して(国際会議の報告) , 2015 .

[56]  Kaisa Miettinen,et al.  A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods , 2015, Structural and Multidisciplinary Optimization.

[57]  Peter Tiño,et al.  Scaling Up Estimation of Distribution Algorithms for Continuous Optimization , 2011, IEEE Transactions on Evolutionary Computation.

[58]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[59]  Xin Yao,et al.  Evolutionary programming using mutations based on the Levy probability distribution , 2004, IEEE Transactions on Evolutionary Computation.

[60]  Mengjie Zhang,et al.  Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems , 2014, IEEE Transactions on Evolutionary Computation.

[61]  Xin Yao,et al.  Estimation of the Distribution Algorithm With a Stochastic Local Search for Uncertain Capacitated Arc Routing Problems , 2016, IEEE Transactions on Evolutionary Computation.

[62]  Sean Luke,et al.  Time-dependent Collaboration Schemes for Cooperative Coevolutionary Algorithms , 2005, AAAI Fall Symposium: Coevolutionary and Coadaptive Systems.

[63]  Peter Tino,et al.  IEEE Transactions on Neural Networks , 2009 .

[64]  Shahryar Rahnamayan,et al.  Metaheuristics in large-scale global continues optimization: A survey , 2015, Inf. Sci..

[65]  Xiaodong Li,et al.  Cooperative Co-evolution with delta grouping for large scale non-separable function optimization , 2010, IEEE Congress on Evolutionary Computation.

[66]  Xiaodong Li,et al.  A Competitive Divide-and-Conquer Algorithm for Unconstrained Large-Scale Black-Box Optimization , 2016, ACM Trans. Math. Softw..

[67]  Xin Yao,et al.  Negatively Correlated Search , 2015, IEEE Journal on Selected Areas in Communications.

[68]  Zhou Wu,et al.  Adaptive multi-context cooperatively coevolving particle swarm optimization for large-scale problems , 2017, Soft Comput..

[69]  David E. Goldberg,et al.  Linkage Identification by Non-monotonicity Detection for Overlapping Functions , 1999, Evolutionary Computation.

[70]  Zhijian Wu,et al.  Enhanced opposition-based differential evolution for solving high-dimensional continuous optimization problems , 2011, Soft Comput..

[71]  Xiaodong Li,et al.  Cooperative Co-Evolution With Differential Grouping for Large Scale Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[72]  Xin Yao,et al.  Meta-Heuristic Algorithms in Car Engine Design: A Literature Survey , 2015, IEEE Transactions on Evolutionary Computation.