Large-Flip Importance Sampling

We propose a new Monte Carlo algorithm for complex discrete distributions. The algorithm is motivated by the N-Fold Way, which is an ingenious event-driven MCMC sampler that avoids rejection moves at any specific state. The N-Fold Way can however get "trapped" in cycles. We surmount this problem by modifying the sampling process. This correction does introduce bias, but the bias is subsequently corrected with a carefully engineered importance sampler.

[1]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[2]  Gerard T. Barkema,et al.  Monte Carlo Methods in Statistical Physics , 1999 .

[3]  A. B. Bortz,et al.  A new algorithm for Monte Carlo simulation of Ising spin systems , 1975 .

[4]  C. Jarzynski.  Nonequilibrium Equality for Free Energy Differences , 1996, cond-mat/9610209.

[5]  Eytan Domany,et al.  Semi-Supervised Learning -- A Statistical Physics Approach , 2006, ArXiv.

[6]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[7]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[8]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[9]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[10]  Wang,et al.  Nonuniversal critical dynamics in Monte Carlo simulations. , 1987, Physical review letters.

[11]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[12]  Éric D. Taillard,et al.  Robust taboo search for the quadratic assignment problem , 1991, Parallel Comput..

[13]  Nando de Freitas,et al.  Conditional mean field , 2006, NIPS.

[14]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[15]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[16]  Thomas Stützle,et al.  Iterated Robust Tabu Search for MAX-SAT , 2003, Canadian Conference on AI.

[17]  Yukito Iba.  EXTENDED ENSEMBLE MONTE CARLO , 2001 .

[18]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[19]  Jun S. Liu,et al.  Bayesian Clustering with Variable and Transformation Selections , 2003 .

[20]  S J Mitchell,et al.  Rejection-free Monte Carlo algorithms for models with continuous degrees of freedom. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[21]  Kotagiri Ramamohanarao,et al.  Sparse Bayesian Learning for Regression and Classification using Markov Chain Monte Carlo , 2002, ICML.

[22]  Mark Jerrum,et al.  The Swendsen-Wang process does not always mix rapidly , 1997, STOC '97.

[23]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..

[24]  Jian Cheng,et al.  AIS-BN: An Adaptive Importance Sampling Algorithm for Evidential Reasoning in Large Bayesian Networks , 2000, J. Artif. Intell. Res..