Recombination operators and selection strategies for evolutionary Markov Chain Monte Carlo algorithms

Markov Chain Monte Carlo (MCMC) methods are often used to sample from intractable target distributions. Some MCMC variants aim to improve the performance by running a population of MCMC chains. In this paper, we investigate the use of techniques from Evolutionary Computation (EC) to design population-based MCMC algorithms that exchange useful information between the individual chains. We investigate how one can ensure that the resulting class of algorithms, called Evolutionary MCMC (EMCMC), samples from the target distribution as expected from any MCMC algorithm. We analytically and experimentally show—using examples from discrete search spaces—that the proposed EMCMCs can outperform standard MCMCs by exploiting common partial structures between the more likely individual states. The MCMC chains in the population interact through recombination and selection. We analyze the required properties of recombination operators and acceptance (or selection) rules in EMCMCs. An important issue is how to preserve the detailed balance property which is a sufficient condition for an irreducible and aperiodic EMCMC to converge to a given target distribution. Transferring EC techniques to population-based MCMCs should be done with care. For instance, we prove that EMCMC algorithms with an elitist acceptance rule do not sample the target distribution correctly.

[1]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[2]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[3]  Cajo J. F. ter Braak,et al.  A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces , 2006, Stat. Comput..

[4]  Nando de Freitas,et al.  An Introduction to MCMC for Machine Learning , 2004, Machine Learning.

[5]  C.J.F. ter Braak,et al.  Genetic algorithms and Markov Chain Monte Carlo: Differential Evolution Markov Chain makes Bayesian computing easy (revised) , 2004 .

[6]  C. Braak,et al.  Genetic algorithms and Markov Chain Monte Carlo: Differential Evolution Markov Chain makes Bayesian computing easy , 2004 .

[7]  Nicholas J. Radcliffe,et al.  Forma Analysis and Random Respectful Recombination , 1991, ICGA.

[8]  Jukka Corander,et al.  Parallell interacting MCMC for learning of topologies of graphical models , 2008, Data Mining and Knowledge Discovery.

[9]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[10]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[11]  S. Q. s3idChMn,et al.  Evolutionary Monte Carlo: Applications to C_p Model Sampling and Change Point Problem , 2000 .

[12]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[13]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[14]  Kathryn B. Laskey,et al.  Population Markov Chain Monte Carlo , 2004, Machine Learning.

[15]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[16]  Malcolm J. A. Strens,et al.  Evolutionary MCMC Sampling and Optimization in Discrete Spaces , 2003, ICML.

[17]  Byoung-Tak Zhang,et al.  System identification using evolutionary Markov chain Monte Carlo , 2001, J. Syst. Archit..

[18]  Joseph Geraci,et al.  A new connection between quantum circuits, graphs and the Ising partition function , 2008, Quantum Inf. Process..

[19]  Fabien Campillo,et al.  Parallel and interacting Markov chain Monte Carlo algorithm , 2009, Math. Comput. Simul..

[20]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[21]  Shumeet Baluja,et al.  Using Optimal Dependency-Trees for Combinational Optimization , 1997, ICML.

[22]  Dirk Thierens,et al.  Elitist recombination: an integrated selection recombination GA , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[23]  Paul Fearnhead.  Editorial: Special issue on adaptive Monte Carlo methods , 2008, Stat. Comput..

[24]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[25]  D. Wolpert,et al.  Adaptive Metropolis Sampling and Optimization with Product Distributions , 2005 .

[26]  David E. Goldberg,et al.  Parallel Recombinative Simulated Annealing: A Genetic Algorithm , 1995, Parallel Comput..

[27]  Fernando G. Lobo,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[28]  Dirk Thierens,et al.  Evolutionary Markov Chain Monte Carlo , 2003, Artificial Evolution.

[29]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[30]  Faming Liang,et al.  EVOLUTIONARY MONTE CARLO: APPLICATIONS TO Cp MODEL SAMPLING AND CHANGE POINT PROBLEM , 2000 .

[31]  Dirk Thierens,et al.  Recombinative EMCMC algorithms , 2005, 2005 IEEE Congress on Evolutionary Computation.