Bandit-based cooperative coevolution for tackling contribution imbalance in large-scale optimization problems

Abstract This paper addresses the issue of computational resource allocation within the context of cooperative coevolution. Cooperative coevolution typically works by breaking a problem down into smaller subproblems (or components) and coevolving them in a round-robin fashion, resulting in a uniform resource allocation among its components. Despite its success on a wide range of problems, cooperative coevolution struggles to perform efficiently when its components do not contribute equally to the overall objective value. This is of crucial importance on large-scale optimization problems where such difference are further magnified. To resolve this imbalance problem, we extend the standard cooperative coevolution to a new generic framework capable of learning the contribution of each component using multi-armed bandit techniques. The new framework allocates the computational resources to each component proportional to their contributions towards improving the overall objective value. This approach results in a more economical use of the limited computational resources. We study different aspects of the proposed framework in the light of extensive experiments. Our empirical results confirm that even a simple bandit-based credit assignment scheme can significantly improve the performance of cooperative coevolution on large-scale continuous problems, leading to competitive performance as compared to the state-of-the-art algorithms.

[1]  Xiaodong Li,et al.  Efficient Resource Allocation in Cooperative Co-Evolution for Large-Scale Global Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[2]  Antonio LaTorre,et al.  Large scale global optimization: Experimental results with MOS-based hybrid algorithms , 2013, 2013 IEEE Congress on Evolutionary Computation.

[3]  Xiaodong Li,et al.  DG2: A Faster and More Accurate Differential Grouping for Large-Scale Black-Box Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Nguyen Xuan Hoai,et al.  Evolutionary Operator Self-adaptation with Diverse Operators , 2012, EuroGP.

[6]  Michèle Sebag,et al.  Analysis of adaptive operator selection techniques on the royal road and long k-path problems , 2009, GECCO.

[7]  Xin Yao,et al.  Dynamic Selection of Evolutionary Algorithm Operators Based on Online Learning and Fitness Landscape Metrics , 2014, SEAL.

[8]  Graham Kendall,et al.  A Classification of Hyper-heuristic Approaches , 2010 .

[9]  A. Kai Qin,et al.  A review of population initialization techniques for evolutionary algorithms , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[10]  Mehryar Mohri,et al.  Multi-armed Bandit Algorithms and Empirical Evaluation , 2005, ECML.

[11]  Michèle Sebag,et al.  Comparison-Based Adaptive Strategy Selection with Bandits in Differential Evolution , 2010, PPSN.

[12]  Shahryar Rahnamayan,et al.  Multilevel framework for large-scale global optimization , 2017, Soft Comput..

[13]  Shahryar Rahnamayan,et al.  Metaheuristics in large-scale global continues optimization: A survey , 2015, Inf. Sci..

[14]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[15]  Michèle Sebag,et al.  Toward comparison-based adaptive operator selection , 2010, GECCO '10.

[16]  Hui Li,et al.  Adaptive strategy selection in differential evolution for numerical optimization: An empirical study , 2011, Inf. Sci..

[17]  Julio R. Banga,et al.  A cooperative strategy for parameter estimation in large scale systems biology models , 2012, BMC Systems Biology.

[18]  Qingfu Zhang,et al.  Adaptive Operator Selection With Bandits for a Multiobjective Evolutionary Algorithm Based on Decomposition , 2014, IEEE Transactions on Evolutionary Computation.

[19]  Xiaodong Li,et al.  A sensitivity analysis of contribution-based cooperative co-evolutionary algorithms , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[20]  Michèle Sebag,et al.  Analyzing bandit-based adaptive operator selection mechanisms , 2010, Annals of Mathematics and Artificial Intelligence.

[21]  Xiaodong Li,et al.  Effects of population initialization on differential evolution for large scale optimization , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[22]  Antonio LaTorre,et al.  Multiple Offspring Sampling in Large Scale Global Optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[23]  Xiaodong Li,et al.  Cooperative Co-Evolution With Differential Grouping for Large Scale Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[24]  Francisco Herrera,et al.  MA-SW-Chains: Memetic algorithm based on local search chains for large scale continuous global optimization , 2010, IEEE Congress on Evolutionary Computation.

[25]  Xiaodong Li,et al.  Designing benchmark problems for large-scale continuous optimization , 2015, Inf. Sci..

[26]  Giuseppe A. Trunfio,et al.  Adaptation in Cooperative Coevolutionary Optimization , 2015 .

[27]  Xiaodong Li,et al.  CBCC3 — A contribution-based cooperative co-evolutionary algorithm with improved exploration/exploitation balance , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[28]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[29]  Shahryar Rahnamayan,et al.  Cooperative co-evolution with sensitivity analysis-based budget assignment strategy for large-scale global optimization , 2017, Applied Intelligence.

[30]  Xiaodong Li,et al.  Effective decomposition of large-scale separable continuous functions for cooperative co-evolutionary algorithms , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[31]  Janez Brest,et al.  Self-adaptive differential evolution algorithm using population size reduction and three strategies , 2011, Soft Comput..

[32]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[33]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[34]  T. Warren Liao,et al.  Three improved hybrid metaheuristic algorithms for engineering design optimization , 2013, Appl. Soft Comput..

[35]  J. Bather,et al.  Multi‐Armed Bandit Allocation Indices , 1990 .

[36]  Xiaodong Li,et al.  Smart use of computational resources based on contribution for cooperative co-evolutionary algorithms , 2011, GECCO '11.

[37]  Xiaodong Li,et al.  A novel hybridization of opposition-based learning and cooperative co-evolutionary for large-scale optimization , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[38]  Francisco Herrera,et al.  A Walk into Metaheuristics for Engineering Optimization: Principles, Methods and Recent Trends , 2015, Int. J. Comput. Intell. Syst..

[39]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[40]  Christian Gagné,et al.  Sustainable cooperative coevolution with a multi-armed bandit , 2013, GECCO '13.