RASH: A Self-adaptive Random Search Method

A variation of an adaptive random search algorithm for the optimization of functions of continuous variables is presented. The scheme does not require any assumptions about the function to be optimized, apart from the availability of evaluations at selected test points. The main design criterion of the Reactive Affine Shaker (RASH) scheme consists of the adaptation of a search region by an affine transformation. The modification takes into account the local knowledge derived from trial points generated with a uniform probability in the search region. The aim is to scout for local minima in the attraction basin where the initial point falls, by adapting the step size and direction to maintain heuristically the largest possible movement per function evaluation. The design is complemented by the analysis of some strategic choices, like the double-shot strategy and the initialization, and by experimental results showing that, in spite of its simplicity, RASH is a promising building block to consider for the development of more complex optimization algorithms.

[1]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..

[2]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[3]  Patrick Siarry,et al.  Tabu Search applied to global optimization , 2000, Eur. J. Oper. Res..

[4]  Patrick Siarry,et al.  Enhanced simulated annealing for globally minimizing functions of many-continuous variables , 1997, TOMS.

[5]  Giampietro Tecchiolli,et al.  On random minimization of functions , 2004, Biological Cybernetics.

[6]  H. Zimmermann Towards global optimization 2: L.C.W. DIXON and G.P. SZEGÖ (eds.) North-Holland, Amsterdam, 1978, viii + 364 pages, US $ 44.50, Dfl. 100,-. , 1979 .

[7]  Roberto Battiti,et al.  Learning with first, second, and no derivatives: A case study in high energy physics , 1994, Neurocomputing.

[8]  A. C. Tsoi,et al.  Improved Simulated Annealing Technique , 1988, Proceedings of the 1988 IEEE International Conference on Systems, Man, and Cybernetics.

[9]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[10]  Sandro Ridella,et al.  Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithmCorrigenda for this article is available here , 1987, TOMS.

[11]  Panos M. Pardalos,et al.  Handbook of applied optimization , 2002 .

[12]  Bart Selman,et al.  Algorithm portfolios , 2001, Artif. Intell..

[13]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[17]  Roberto Battiti,et al.  The Reactive Tabu Search , 1994, INFORMS J. Comput..