Adaptive Learning Search, a New Tool to Help Comprehending Metaheuristics

The majority of the algorithms used to solve hard optimization problems today are population metaheuristics. These methods are often presented under a purely algorithmic angle, while insisting on the metaphors which led to their design. We propose in this article to regard population metaheuristics as methods making evolution a probabilistic sampling of the objective function, either explicitly, implicitly, or directly, via processes of learning, diversification, and intensification. We present a synthesis of some metaheuristics and their functioning seen under this angle, called Adaptive Learning Search. We discuss how to design metaheuristics following this approach, and propose an implementation with our Open Metaheuristics framework, along with concrete examples.

[1]  Shumeet Baluja,et al.  Genetic Algorithms and Explicit Search Statistics , 1996, NIPS.

[2]  M. Resende,et al.  GREEDY RANDOMIZED ADAPTIVE SEARCH PROCEDURES (GRASP) , 1999 .

[3]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[4]  Ivar Jacobson,et al.  The unified modeling language reference manual , 2010 .

[5]  G. P. McKeown,et al.  Optimization Software Class Libraries , 2002, Operations Research/Computer Science Interfaces Series.

[6]  János Kertész,et al.  Advances in Computer Simulation , 1998 .

[7]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[8]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[9]  S. Q. s3idChMn,et al.  Evolutionary Monte Carlo: Applications to C_p Model Sampling and Change Point Problem , 2000 .

[10]  Kalyanmoy Deb,et al.  Special Session on Real-Parameter Optimization at CEC 05 , 2005 .

[11]  Ralph E. Johnson,et al.  Design Patterns: Abstraction and Reuse of Object-Oriented Design , 1993, ECOOP.

[12]  Robyn A. Owens,et al.  Registration of stereo and temporal images of the retina , 1999, IEEE Transactions on Medical Imaging.

[13]  P. Bosman,et al.  IDEAs based on the normal kernels probability density function , 2000 .

[14]  Y. Okamoto,et al.  Thermodynamics of Helix-Coil Transitions Studied by Multicanonical Algorithms , 1995, chem-ph/9505006.

[15]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[16]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[17]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[18]  P. M. C. Oliveira Broad histogram: an overview , 2000, cond-mat/0003300.

[19]  Ehl Emile Aarts,et al.  Statistical cooling : a general approach to combinatorial optimization problems , 1985 .

[20]  Stefan Voß,et al.  Generic metaheuristics application to industrial engineering problems , 1999 .

[21]  G. Harik Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[22]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[23]  Marco Dorigo,et al.  Distributed Optimization by Ant Colonies , 1992 .

[24]  Oliver Wendt,et al.  Cooperative Simulated Annealing: How much cooperation is enough ? , 1998 .

[25]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[26]  Raphaël Dorne,et al.  HSF: the iOpt's framework to easily design metaheuristic methods , 2004 .

[27]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[28]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[29]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[30]  S. Dreyfus,et al.  Thermodynamical Approach to the Traveling Salesman Problem : An Efficient Simulation Algorithm , 2004 .

[31]  Martin Stuart Jones An object-oriented framework for the implementation of search techniques , 2000 .

[32]  P. Bosman,et al.  An algorithmic framework for density estimation based evolutionary algorithms , 1999 .

[33]  John H. Holland,et al.  Outline for a Logical Theory of Adaptive Systems , 1962, JACM.

[34]  Ralph Johnson,et al.  Design patterns: elements of reuseable object-oriented software , 1994 .

[35]  Johann Dréo,et al.  Robust rigid registration of retinal angiograms through optimization , 2006, Comput. Medical Imaging Graph..

[36]  Yukito Iba Population Annealing An approach to finite-temperature calculation , 2003 .

[37]  Maarten Keijzer,et al.  Evolving Objects: A General Purpose Evolutionary Computation Library , 2001, Artificial Evolution.

[38]  Éric D. Taillard,et al.  Mic2003: the Fifth Metaheuristics International Conference a Statistical Test for Comparing Success Rateséric , 2022 .

[39]  Johann Dréo,et al.  Metaheuristics for Hard Optimization: Methods and Case Studies , 2005 .

[40]  Faming Liang,et al.  EVOLUTIONARY MONTE CARLO: APPLICATIONS TO Cp MODEL SAMPLING AND CHANGE POINT PROBLEM , 2000 .

[41]  Patrick Siarry,et al.  A theoretical study on the behavior of simulated annealing leading to a new cooling schedule , 2005, Eur. J. Oper. Res..

[42]  E. J. NewmanSanta Error estimation in the histogram Monte Carlo method , 1998 .

[43]  P. Bosman,et al.  Continuous iterated density estimation evolutionary algorithms within the IDEA framework , 2000 .

[44]  Michael Creutz,et al.  Microcanonical Monte Carlo Simulation , 1983 .

[45]  Nicolas Monmarché,et al.  On the similarities between AS, BSC and PBIL: toward the birth of a new meta-heuristic , 1999 .

[46]  Luca Maria Gambardella,et al.  Adaptive memory programming: A unified view of metaheuristics , 1998, Eur. J. Oper. Res..

[47]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[48]  Shumeet Baluja,et al.  Fast Probabilistic Modeling for Combinatorial Optimization , 1998, AAAI/IAAI.

[49]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[50]  Alan M. Ferrenberg,et al.  Optimized Monte Carlo data analysis. , 1989, Physical Review Letters.

[51]  Johann Dréo,et al.  Retinal angiogram registration by estimation of distribution algorithm , 2006 .

[52]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[53]  Gilbert Syswerda,et al.  Simulated Crossover in Genetic Algorithms , 1992, FOGA.