Variational methods are becoming increasingly popular for the approximate solution of complex probabilistic models in machine learning, computer vision, information retrieval and many other fields. Unfortunately, for every new application it is necessary first to derive the specific forms of the variational update equations for the particular probabilistic model being used, and then to implement these equations in applicationspecific software. Each of these steps is both time consuming and error prone. We have therefore recently developed a general purpose inference engine called VIBES [1] (‘Variational Inference for Bayesian Networks’) which allows a wide variety of probabilistic models to be implemented and solved variationally without recourse to coding. New models are specified as a directed acyclic graph using an interface analogous to a drawing package, and VIBES then automatically generates and solves the variational equations. The original version of VIBES assumed a fully factorized variational posterior distribution. In this paper we present an extension of VIBES in which the variational posterior distribution corresponds to a sub-graph of the full probabilistic model. Such structured distributions can produce much closer approximations to the true posterior distribution. We illustrate this approach using an example based on Bayesian hidden Markov models.
[1]
Michael I. Jordan,et al.
Exploiting Tractable Substructures in Intractable Networks
,
1995,
NIPS.
[2]
Michael I. Jordan.
Graphical Models
,
1998
.
[3]
Hagai Attias,et al.
A Variational Bayesian Framework for Graphical Models
,
1999
.
[4]
Charles M. Bishop.
Variational principal components
,
1999
.
[5]
David J. Spiegelhalter,et al.
Probabilistic Networks and Expert Systems
,
1999,
Information Science and Statistics.
[6]
David J. C. MacKay,et al.
Ensemble Learning for Blind Image Separation and Deconvolution
,
2000
.
[7]
Andrew Thomas,et al.
WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility
,
2000,
Stat. Comput..
[8]
Zoubin Ghahramani,et al.
Propagation Algorithms for Variational Bayesian Learning
,
2000,
NIPS.
[9]
Wim Wiegerinck,et al.
Variational Approximations between Mean Field Theory and the Junction Tree Algorithm
,
2000,
UAI.
[10]
David J. Spiegelhalter,et al.
VIBES: A Variational Inference Engine for Bayesian Networks
,
2002,
NIPS.
[11]
Michael I. Jordan,et al.
An Introduction to Variational Methods for Graphical Models
,
1999,
Machine Learning.
[12]
Michael I. Jordan,et al.
Factorial Hidden Markov Models
,
1995,
Machine Learning.