Simulated annealing: Practice versus theory

Simulated annealing (SA) presents an optimization technique with several striking positive and negative features. Perhaps its most salient feature, statistically promising to deliver an optimal solution, in current practice is often spurned to use instead modified faster algorithms, ''simulated quenching'' (SQ). Using the author's Adaptive Simulated Annealing (ASA) code, some examples are given which demonstrate how SQ can be much faster than SA without sacrificing accuracy.

[1]  Karl Heinz Hoffmann,et al.  Concepts in Optimizing Simulated Annealing Schedules: An Adaptive Approach for Parallel and Vector Machines , 1991 .

[2]  L. Ingber Very fast simulated re-annealing , 1989 .

[3]  Kris Vasudevan,et al.  Simulated annealing statics computation using an order-based energy function , 1991 .

[4]  Søren Brunak,et al.  A Travelling Salesman Approach to Protein Conformation , 1989, Complex Syst..

[5]  Petar D. Simic,et al.  Statistical mechanics as the underlying theory of ‘elastic’ and ‘neural’ optimisations , 1990 .

[6]  James P. Kelly,et al.  Using Simulated Annealing to Solve Controlled Rounding Problems , 1990, INFORMS J. Comput..

[7]  Murray Goodman,et al.  Effects of dehydroalanine on peptide conformations , 1992 .

[8]  Cris Koutsougeras,et al.  An annealing approach to associative recall in the CBM model , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[9]  Kuniharu Kishida,et al.  Physical Langevin model and the time-series model in systems far from equilibrium , 1982 .

[10]  Charles V. Packer,et al.  Applying row-column permutation to matrix representations of large citation networks , 1989, Inf. Process. Manag..

[11]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[12]  Mark E. Johnson,et al.  Optimal deployment of missile interceptors , 1988 .

[13]  Daniel H. Rothman,et al.  Nonlinear inversion, statistical mechanics, and residual statics estimation , 1985 .

[14]  David E. Goldberg,et al.  A Note on Boltzmann Tournament Selection for Genetic Algorithms and Population-Oriented Simulated Annealing , 1990, Complex Syst..

[15]  Saul B. Gelfand,et al.  Analysis of simulated annealing type algorithms , 1987 .

[16]  T. H. Sloane,et al.  CMOS leaf-cell design using simulated annealing , 1992, [1992] Proceedings of the 35th Midwest Symposium on Circuits and Systems.

[17]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  D. Bouzida,et al.  A simulated annealing approach for probing biomolecular structures , 1993, [1993] Proceedings of the Twenty-sixth Hawaii International Conference on System Sciences.

[19]  G. Bilbro Efficient generators in simulated annealing , 1991 .

[20]  Sandro Ridella,et al.  Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithmCorrigenda for this article is available here , 1987, TOMS.

[21]  Larry J. Eshelman,et al.  Spurious Correlations and Premature Convergence in Genetic Algorithms , 1990, FOGA.

[22]  E. Bonomi,et al.  The N-City Travelling Salesman Problem: Statistical Mechanics and the Metropolis Algorithm , 1984 .

[23]  Peter M. A. Sloot,et al.  Nonconvex continuous optimization experiments on a transputer system , 1992 .

[24]  Kenneth J. Supowit,et al.  Simulated Annealing Without Rejected Moves , 1986, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[25]  Radford M. Neal Bayesian training of backpropagation networks by the hybrid Monte-Carlo method , 1992 .

[26]  William L. Goffe,et al.  SIMANN: FORTRAN module to perform Global Optimization of Statistical Functions with Simulated Annealing , 1992 .

[27]  John F. Marshall,et al.  Financial Engineering: A Complete Guide to Financial Innovation , 1992 .

[28]  Thomas L. Beck,et al.  Locating stationary paths in functional integrals: An optimization method utilizing the stationary phase Monte Carlo sampling function , 1989 .

[29]  Lester Ingber,et al.  Statistical Mechanics Algorithm for Response to Targets (SMART) , 1985, UAI 1985.

[30]  S. Duane,et al.  Hybrid Monte Carlo , 1987 .

[31]  Martin Pincus,et al.  Letter to the Editor - A Monte Carlo Method for the Approximate Solution of Certain Types of Constrained Optimization Problems , 1970, Oper. Res..

[32]  L. Ingber,et al.  Application of statistical mechanics methodology to term-structure bond-pricing models , 1991 .

[33]  M. E. Johnson,et al.  Generalized simulated annealing for function optimization , 1986 .

[34]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[35]  John E. Savage,et al.  Parallelism in Graph-Partitioning , 1991, J. Parallel Distributed Comput..

[36]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[37]  Kimmo Kaski,et al.  Image Deconvolution with Simulated Annealing Method , 1990 .

[38]  Ingber Generic mesoscopic neural networks based on statistical mechanics of neocortical interactions. , 1992, Physical review. A, Atomic, molecular, and optical physics.

[39]  N. E. Collins,et al.  Simulated annealing - an annotated bibliography , 1988 .

[40]  Catherine A. Schevon,et al.  Optimization by simulated annealing: An experimental evaluation , 1984 .

[41]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[42]  Carsten Peterson,et al.  Neural Networks and NP-complete Optimization Problems; A Performance Study on the Graph Bisection Problem , 1988, Complex Syst..

[43]  Gregory J. Wolff,et al.  Deterministic Boltzmann machine VLSI can be scaled using multi-chip modules , 1992, [1992] Proceedings of the International Conference on Application Specific Array Processors.

[44]  Kuniharu Kishida,et al.  Equivalent random force and time‐series model in systems far from equilibrium , 1984 .

[45]  N Ohyama,et al.  Analysis of the cost function used in simulated annealing for CT image reconstruction. , 1990, Applied optics.

[46]  Michael F. Wehner,et al.  Mathematical comparison of combat computer models to exercise data , 1991 .

[47]  C. DeWitt-Morette,et al.  Techniques and Applications of Path Integration , 1981 .

[48]  Cecilia R. Aragon,et al.  Optimization by Simulated Annealing: An Experimental Evaluation; Part I, Graph Partitioning , 1989, Oper. Res..

[49]  J. Powell Mathematical Methods in Physics , 1965 .

[50]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[51]  S Brunak,et al.  Multiple alignment using simulated annealing: branch point definition in human mRNA splicing. , 1992, Nucleic acids research.

[52]  Mrinal K. Sen,et al.  Nonlinear one-dimensional seismic waveform inversion using simulated annealing , 1991 .

[53]  R. Bucy,et al.  Decision tree design by simulated annealing , 1993 .

[54]  A. Frigessi,et al.  Convergence of Some Partially Parallel Gibbs Samplers with Annealing , 1993 .

[55]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[56]  Petar D. Simic Constrained Nets for Graph Matching and Other Quadratic Assignment Problems , 1991, Neural Comput..

[57]  Michael J. Black,et al.  Robust dynamic motion estimation over time , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[58]  B. Rosen,et al.  Function Optimization Based On Advanced Simulated Annealing , 1992, Workshop on Physics and Computation.

[59]  L. Ingber Adaptive Simulated Annealing (ASA) , 1993 .

[60]  M. D. Collins,et al.  Optimal time-domain beamforming with simulated annealing including application of a priori information , 1990 .

[61]  G. Toulouse,et al.  Ultrametricity for physicists , 1986 .

[62]  V. Cerný Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm , 1985 .

[63]  Ingber,et al.  Statistical-mechanical aids to calculating term-structure models. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[64]  Kenneth Lange,et al.  Applications of coding theory to the design of somatic cell hybrid panels , 1988, Mathematical Biosciences.

[65]  Richard C. Dubes,et al.  Experiments in projection and clustering by simulated annealing , 1989, Pattern Recognit..

[66]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[67]  Abraham Charnes,et al.  Extended Pincus Theorems and Convergence of Simulated Annealing , 1989 .

[68]  Peter B. Gibbons,et al.  The construction of subsquare free Latin squares by simulated annealing , 1992, Australas. J Comb..

[69]  Bruce E. Rosen,et al.  Genetic Algorithms and Very Fast Simulated Reannealing: A comparison , 1992 .

[70]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[71]  A. Jagota Efficiently approximating max-clique in a Hopfield-style network , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[72]  D. Sworder,et al.  Statistical mechanics of combat with human factors , 1991 .

[73]  Ingber,et al.  Statistical mechanics of neocortical interactions: A scaling paradigm applied to electroencephalography. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[74]  Jacques J. Vidal,et al.  Image Recognition and Reconstruction Using Associative Magnetic Processing , 1992, Int. J. Pattern Recognit. Artif. Intell..

[75]  Lester Ingber,et al.  ASA-README included with ASA code , 1993 .

[76]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[77]  David L. Dowe,et al.  A decision graph explanation of protein secondary structure prediction , 1993, [1993] Proceedings of the Twenty-sixth Hawaii International Conference on System Sciences.

[78]  Z. Michalewicz,et al.  Genetic algorithms for numerical optimization , 1991 .

[79]  David E. Goldberg,et al.  Parallel Recombinative Simulated Annealing: A Genetic Algorithm , 1995, Parallel Comput..

[80]  L N Frazer,et al.  Rapid Determination of the Critical Temperature in Simulated Annealing Inversion , 1990, Science.

[81]  Kennetb A. De Genetic Algorithms Are NOT Function Optimizers , 1992 .

[82]  E. Tirapegui,et al.  Functional Integration and Semiclassical Expansions , 1982 .

[83]  Wesley E. Snyder,et al.  Optimization by Mean Field Annealing , 1988, NIPS.