In this work we apply statistical learning methods in the context of combinatorial optimization, which is understood as finding a binary string minimizing a given cost function. We first consider probability densities over binary strings and we define two different statistical criteria. Then we recast the initial problem as the problem of finding a density minimizing one of the two criteria. We restrict ourselves to densities described by a small number of parameters and solve the new problem by means of gradient techniques. This results in stochastic algorithms which iteratively update density parameters. We apply these algorithms to two families of densities, the Bernoulli model and the Gaussian model. The algorithms have been implemented and some experiments are reported.
Richard S. Sutton,et al.
Neuronlike adaptive elements that can solve difficult learning control problems
IEEE Transactions on Systems, Man, and Cybernetics.
C. D. Gelatt,et al.
Optimization by Simulated Annealing
J J Hopfield,et al.
Neurons with graded response have collective computational properties like those of two-state neurons.
Proceedings of the National Academy of Sciences of the United States of America.
Donald Geman,et al.
Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
IEEE Transactions on Pattern Analysis and Machine Intelligence.
Esther Levin,et al.
A statistical approach to learning and generalization in layered neural networks
Kumpati S. Narendra,et al.
Learning automata - an introduction
Michael I. Jordan,et al.
Reinforcement Learning by Probability Matching
Rich Caruana,et al.
Removing the Genetics from the Standard Genetic Algorithm
Yuval Rabani,et al.
A computational view of population genetics
Mark Jerrum,et al.
The Markov chain Monte Carlo method: an approach to approximate counting and integration
Zbigniew Michalewicz,et al.
Genetic Algorithms + Data Structures = Evolution Programs
Springer Berlin Heidelberg.
Paul A. Viola,et al.
MIMIC: Finding Optima by Estimating Probability Densities
Dorit S. Hochbaum,et al.
Approximation Algorithms for NP-Hard Problems
Thomas Bäck,et al.
Evolutionary computation: comments on the history and current state
IEEE Trans. Evol. Comput..