Distributed Parameter Estimation in Probabilistic Graphical Models

This paper presents foundational theoretical results on distributed parameter estimation for undirected probabilistic graphical models. It introduces a general condition on composite likelihood decompositions of these models which guarantees the global consistency of distributed estimators, provided the local estimators are consistent.

[1]  Qiang Liu,et al.  Distributed Parameter Estimation via Pseudo-likelihood , 2012, ICML.

[2]  Misha Denil,et al.  Linear and Parallel Learning of Markov Random Fields , 2013, ICML.

[3]  Joseph K. Bradley,et al.  Sample Complexity of Composite Likelihood , 2012, AISTATS.

[4]  Alfred O. Hero,et al.  Distributed Learning of Gaussian Graphical Models via Marginal Likelihoods , 2013, AISTATS.

[5]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[6]  Sebastian Nowozin.  Constructing Composite Likelihoods in General Random Fields , 2013 .

[7]  Nando de Freitas,et al.  Asymptotic Efficiency of Deterministic Estimators for Discrete Energy-Based Models: Ratio Matching and Pseudolikelihood , 2011, UAI.

[8]  Stephen E. Fienberg,et al.  Maximum likelihood estimation in log-linear models , 2011, 1104.3618.

[9]  John Odentrantz,et al.  Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues , 2000, Technometrics.

[10]  N. Reid,et al.  AN OVERVIEW OF COMPOSITE LIKELIHOOD METHODS , 2011 .

[11]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[12]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[13]  Nando de Freitas,et al.  Inductive Principles for Restricted Boltzmann Machine Learning , 2010, AISTATS.

[14]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[15]  Charles J. Geyer,et al.  EXTENDING PSEUDO-LIKELIHOOD FOR POTTS MODELS , 2011 .

[16]  J. M. Hammersley,et al.  Markov fields on finite graphs and lattices , 1971 .

[17]  Padhraic Smyth,et al.  Learning with Blocks: Composite Likelihood and Contrastive Divergence , 2010, AISTATS.

[18]  D. Griffeath,et al.  Introduction to Random Fields , 2020, 2007.09660.

[19]  Alfred O. Hero,et al.  Distributed Covariance Estimation in Gaussian Graphical Models , 2010, IEEE Transactions on Signal Processing.

[20]  Michael I. Jordan,et al.  An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators , 2008, ICML '08.

[21]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[22]  Guy Lebanon,et al.  Stochastic Composite Likelihood , 2010, J. Mach. Learn. Res..

[23]  Alfred O. Hero,et al.  Marginal Likelihoods for Distributed Parameter Estimation of Gaussian Graphical Models , 2014, IEEE Transactions on Signal Processing.

[24]  K. Mardia,et al.  Maximum likelihood estimation using composite likelihoods for closed exponential families , 2009 .

[25]  Harry Joe,et al.  Composite Likelihood Methods , 2012 .

[26]  D. J. Strauss,et al.  Pseudolikelihood Estimation for Social Networks , 1990 .