Field-Theoretic Methods for Intractable Probabilistic Models

We describe a general technique for estimating the intractable quantities that occur in a wide variety of largescale probabilistic models. The technique transforms intractable sums into integrals which are subsequently approximated via saddle point methods. When applied to sigmoid and noisy-OR networks, the technique yields a generic mean-field approximation as well as a second order Gaussian approximation that accounts for the pairwise correlations between random variables in the network. In two example models, we observe that our lowest order approximation is identical to expressions obtained using Plefka’s approach for deriving the TAP equations.