UvA-DARE (Digital Academic Repository) Generalized belief propagation on tree robust structured region graphs Generalized Belief Propagation on Tree Robust Structured Region Graphs

This paper provides some new guidance in the construction of region graphs for Generalized Belief Propagation (GBP). We connect the problem of choosing the outer regions of a Loop-Structured Region Graph (SRG) to that of finding a fundamental cycle basis of the corresponding Markov network. We also define a new class of tree-robust Loop-SRG for which GBP on any induced (spanning) tree of the Markov network, obtained by setting to zero the off-tree interactions, is exact. This class of SRG is then mapped to an equivalent class of tree-robust cycle bases on the Markov network. We show that a tree-robust cycle basis can be identified by proving that for every subset of cycles, the graph obtained from the edges that participate in a single cycle only, is multiply connected. Using this we identify two classes of tree-robust cycle bases: planar cycle bases and “star” cycle bases. In experiments we show that tree-robustness can be suc-cessfully exploited as a design principle to improve the accuracy and convergence of GBP.

[1]  R. Kikuchi.  A Theory of Cooperative Phenomena , 1951 .

[2]  T. Morita.  Consistent relations in the method of reducibility in the cluster variation method , 1984 .

[3]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[4]  T. Morita.  Formal Structure of the Cluster Variation Method , 1994 .

[5]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[6]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[7]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[8]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[9]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[10]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[11]  W. Freeman,et al.  Bethe free energy, Kikuchi approximations, and belief propagation algorithms , 2001 .

[12]  Yee Whye Teh,et al.  Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation , 2001, UAI.

[13]  Rina Dechter,et al.  Iterative Join-Graph Propagation , 2002, UAI.

[14]  Alan L. Yuille,et al.  CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation , 2002, Neural Computation.

[15]  T. Heskes.  Stable Fixed Points of Loopy Belief Propagation Are Minima of the Bethe Free Energy , 2002 .

[16]  Martin J. Wainwright,et al.  Semidefinite Relaxations for Approximate Inference on Graphs with Cycles , 2003, NIPS.

[17]  Robert J. McEliece,et al.  Belief Propagation on Partially Ordered Sets , 2003, Mathematical Systems Theory in Biology, Communications, Computation, and Finance.

[18]  John W. Fisher,et al.  Message Errors in Belief Propagation , 2004, NIPS.

[19]  Max Welling,et al.  On the Choice of Regions for Generalized Belief Propagation , 2004, UAI.

[20]  Hilbert J. Kappen,et al.  Sufficient Conditions for Convergence of Loopy Belief Propagation , 2005, UAI.

[21]  Yee Whye Teh,et al.  Structured Region Graphs: Morphing EP into GBP , 2005, UAI.

[22]  Payam Pakzad,et al.  Estimation and Marginalization Using the Kikuchi Approximation Methods , 2005, Neural Computation.

[23]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[24]  Martin J. Wainwright,et al.  A new class of upper bounds on the log partition function , 2002, IEEE Transactions on Information Theory.

[25]  Adnan Darwiche,et al.  An Edge Deletion Semantics for Belief Propagation and its Practical Impact on Approximation Quality , 2006, AAAI.

[26]  Ian McGraw,et al.  Residual Belief Propagation: Informed Scheduling for Asynchronous Message Passing , 2006, UAI.

[27]  Tom Heskes,et al.  Convexity Arguments for Efficient Minimization of the Bethe and Kikuchi Free Energies , 2006, J. Artif. Intell. Res..

[28]  Tommi S. Jaakkola,et al.  New Outer Bounds on the Marginal Polytope , 2007, NIPS.

[29]  Tommi S. Jaakkola,et al.  Convergent Propagation Algorithms via Oriented Trees , 2007, UAI.

[30]  Tamir Hazan,et al.  Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies , 2008, UAI.

[31]  Amir Globerson,et al.  Convergent message passing algorithms - a unifying view , 2009, UAI.

[32]  Kurt Mehlhorn,et al.  Cycle bases in graphs characterization, algorithms, complexity, and applications , 2009, Comput. Sci. Rev..

[33]  Yong Gao.  The degree distribution of random k-trees , 2009, Theor. Comput. Sci..