Evolutionary Computation Meets Machine Learning: A Survey

Evolutionary computation (EC) is a kind of optimization methodology inspired by the mechanisms of biological evolution and behaviors of living organisms. In the literature, the terminology evolutionary algorithms is frequently treated the same as EC. This article focuses on making a survey of researches based on using ML techniques to enhance EC algorithms. In the framework of an ML-technique enhanced-EC algorithm (MLEC), the main idea is that the EC algorithm has stored ample data about the search space, problem features, and population information during the iterative search process, thus the ML technique is helpful in analyzing these data for enhancing the search performance. The paper presents a survey of five categories: ML for population initialization, ML for fitness evaluation and selection, ML for population reproduction and variation, ML for algorithm adaptation, and ML for local search.

[1]  Andreas Zell,et al.  A Clustering Based Niching Method for Evolutionary Algorithms , 2003, GECCO.

[2]  Hitoshi Iba,et al.  Accelerating Differential Evolution Using an Adaptive Local Search , 2008, IEEE Transactions on Evolutionary Computation.

[3]  J. Kennedy Stereotyping: improving particle swarm performance with cluster analysis , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[4]  Jing J. Liang,et al.  Adaptive Comprehensive Learning Particle Swarm Optimizer with History Learning , 2006, SEAL.

[5]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.

[6]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[7]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[8]  Chee Keong Kwoh,et al.  Feasibility Structure Modeling: An Effective Chaperone for Constrained Memetic Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[9]  Shinn-Ying Ho,et al.  Intelligent evolutionary algorithms for large parameter optimization problems , 2004, IEEE Trans. Evol. Comput..

[10]  Jun Zhang,et al.  A contour method in population-based stochastic algorithms , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[11]  Thomas Bäck,et al.  Evolutionary Algorithms for Real World Applications , 2008 .

[12]  H.S.-H. Chung,et al.  Extended Ant Colony Optimization Algorithm for Power Electronic Circuit Design , 2009, IEEE Transactions on Power Electronics.

[13]  Mengjie Zhang,et al.  A New Crossover Operator in Genetic Programming for Object Classification , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  M. R. Lemes,et al.  Neural-network-assisted genetic algorithm applied to silicon clusters , 2003 .

[15]  Jong-Hwan Kim,et al.  Evolutionary multi-objective optimization in robot soccer system for education , 2009, IEEE Computational Intelligence Magazine.

[16]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[17]  T. Yalcinoz,et al.  Power economic dispatch using a hybrid genetic algorithm , 2001 .

[18]  J. Hunger,et al.  Optimization and analysis of force field parameters by combination of genetic algorithms and neural networks , 1999, J. Comput. Chem..

[19]  Jiang-She Zhang,et al.  A dynamic clustering based differential evolution algorithm for global optimization , 2007, Eur. J. Oper. Res..

[20]  Yew-Soon Ong,et al.  A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[21]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[22]  Xin Yao,et al.  Clustering and learning Gaussian distribution for continuous optimization , 2005, IEEE Trans. Syst. Man Cybern. Part C.

[23]  Justinian P. Rosca,et al.  Learning by adapting representations in genetic programming , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[24]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[25]  Mark Sumner,et al.  A Fast Adaptive Memetic Algorithm for Online and Offline Control Design of PMSM Drives , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Carlos A. Coello Coello,et al.  Using Clustering Techniques to Improve the Performance of a Multi-objective Particle Swarm Optimizer , 2004, GECCO.

[27]  Yew-Soon Ong,et al.  A Probabilistic Memetic Framework , 2009, IEEE Transactions on Evolutionary Computation.

[28]  Aurora Trinidad Ramirez Pozo,et al.  Effective Linkage Learning Using Low-Order Statistics and Clustering , 2009, IEEE Transactions on Evolutionary Computation.

[29]  Ming Li,et al.  Hybrid Evolutionary Search Method Based on Clusters , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Gary G. Yen,et al.  Dynamic Evolutionary Algorithm With Variable Relocation , 2009, IEEE Transactions on Evolutionary Computation.

[31]  Wu Muqing,et al.  An Adaptive LS-SVM Based Differential Evolution Algorithm , 2009, 2009 International Conference on Signal Processing Systems.

[32]  Yew-Soon Ong,et al.  Optinformatics for schema analysis of binary genetic algorithms , 2008, GECCO '08.

[33]  Huang Xi-yue Solving TSP with Characteristic of Clustering by Ant Colony Algorithm , 2004 .

[34]  Mitsuo Gen,et al.  Reliability Optimization Design Using a Hybridized Genetic Algorithm with a Neural-Network Technique , 2001 .

[35]  Jing Lu,et al.  Adaptive evolutionary programming based on reinforcement learning , 2008, Inf. Sci..

[36]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[37]  Sushil J. Louis,et al.  Learning with case-injected genetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[38]  John A. W. McCall,et al.  Using a Markov network as a surrogate fitness function in a genetic algorithm , 2010, IEEE Congress on Evolutionary Computation.

[39]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[40]  A. Kai Qin,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[41]  T. Back,et al.  Evolutionary algorithms for real world applications [Application Notes] , 2008, IEEE Computational Intelligence Magazine.

[42]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[43]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[44]  Yaochu Jin A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[45]  El-Ghazali Talbi,et al.  Using Datamining Techniques to Help Metaheuristics: A Short Survey , 2006, Hybrid Metaheuristics.

[46]  Sung-Bae Cho,et al.  An efficient genetic algorithm with less fitness evaluation by clustering , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[47]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[48]  Jun Zhang,et al.  Adaptive control of acceleration coefficients for particle swarm optimization based on clustering analysis , 2007, 2007 IEEE Congress on Evolutionary Computation.

[49]  Yong Yang,et al.  A Coarse-Grained Parallel Genetic Algorithm Employing Cluster Analysis for Multi-modal Numerical Optimisation , 2003, Artificial Evolution.

[50]  Christine A. Shoemaker,et al.  Local function approximation in evolutionary algorithms for the optimization of costly functions , 2004, IEEE Transactions on Evolutionary Computation.

[51]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[52]  J. Hunger,et al.  Optimization and analysis of force field parameters by combination of genetic algorithms and neural networks , 1999 .

[53]  Sanjay Srivastava,et al.  Neural network embedded multiobjective genetic algorithm to solve non-linear time-cost tradeoff problems of project scheduling , 2008 .

[54]  E Mjolsness,et al.  Machine learning for science: state of the art and future prospects. , 2001, Science.

[55]  Qingfu Zhang,et al.  A surrogate-assisted evolutionary algorithm for minimax optimization , 2010, IEEE Congress on Evolutionary Computation.

[56]  Jun Zhang,et al.  A clustering-based adaptive parameter control method for continuous ant colony optimization , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[57]  Pedro Larrañaga,et al.  Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks , 2005, Evolutionary Computation.

[58]  Anne Auger,et al.  Performance evaluation of an advanced local search evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[59]  A.C. Martinez-Estudillo,et al.  Hybridization of evolutionary algorithms and local search by means of a clustering method , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[60]  Yew-Soon Ong,et al.  Memetic Computation—Past, Present & Future [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[61]  Franz Aurenhammer,et al.  Evolution strategy and hierarchical clustering , 2002 .

[62]  So-Youn Park,et al.  Improvement of a multi-objective differential evolution using clustering algorithm , 2009, 2009 IEEE International Symposium on Industrial Electronics.

[63]  Jacek M. Zurada,et al.  Building virtual community in computational intelligence and machine learning [Research Frontier] , 2009, IEEE Computational Intelligence Magazine.

[64]  Jun Zhang,et al.  Clustering-Based Adaptive Crossover and Mutation Probabilities for Genetic Algorithms , 2007, IEEE Transactions on Evolutionary Computation.

[65]  Jun Zhang,et al.  Orthogonal Learning Particle Swarm Optimization , 2011, IEEE Transactions on Evolutionary Computation.

[66]  Millie Pant,et al.  Differential Evolution using Quadratic Interpolation for Initializing the Population , 2009, 2009 IEEE International Advance Computing Conference.

[67]  Mika Johnsson,et al.  An adaptive hybrid genetic algorithm for the three-matching problem , 2000, IEEE Trans. Evol. Comput..

[68]  Tomonobu Senjyu,et al.  Fast technique for unit commitment by genetic algorithm based on unit clustering , 2005 .

[69]  Yew-Soon Ong,et al.  A Frequent Pattern Mining Algorithm for Understanding Genetic Algorithms , 2008, ICIC.

[70]  Yuping Wang,et al.  An orthogonal genetic algorithm with quantization for global numerical optimization , 2001, IEEE Trans. Evol. Comput..

[71]  Yew-Soon Ong,et al.  Classifier-assisted constrained evolutionary optimization for automated geometry selection of orthodontic retraction spring , 2010, IEEE Congress on Evolutionary Computation.

[72]  Chuan-Kang Ting,et al.  Linkage Discovery through Data Mining [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[73]  Gary G. Yen Learning and Intelligence [Editor's remarks] , 2009, IEEE Comput. Intell. Mag..