Probability Learning and Soft Quantization in Bayesian Factor Graphs

We focus on learning the probability matrix for discrete random variables in factor graphs. We review the problem and its variational approximation and, via entropic priors, we show that soft quantization can be included in a probabilistically-consistent fashion in a factor graph that learns the mutual relationship among the variables involved. The framework is explained with reference the ”Tipper” example and the results of a Matlab simulation are included.

[1]  Charles M. Bishop,et al.  Variational Message Passing , 2005, J. Mach. Learn. Res..

[2]  Vincent Y. F. Tan,et al.  Learning Latent Tree Graphical Models , 2010, J. Mach. Learn. Res..

[3]  Francesco Palmieri,et al.  Entropic priors for short-term stochastic process classification , 2011, 14th International Conference on Information Fusion.

[4]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[5]  John M. Winn,et al.  Variational Message Passing and its Applications , 2004 .

[6]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[7]  Jan Jantzen,et al.  Foundations of fuzzy control , 2007 .

[8]  Francesco Palmieri,et al.  Objective priors from maximum entropy in data classification , 2013, Inf. Fusion.

[9]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[10]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[11]  V. Novák,et al.  Mathematical Principles of Fuzzy Logic , 1999 .

[12]  S. Fienberg.  When did Bayesian inference become "Bayesian"? , 2006 .

[13]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[14]  E. Jaynes.  Probability theory : the logic of science , 2003 .

[15]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Processing Magazine.

[16]  Andrew W. Eckford,et al.  Expectation Maximization as Message Passing - Part I: Principles and Gaussian Messages , 2009, ArXiv.

[17]  Matthew J. Beal,et al.  Variational Bayesian learning of directed graphical models with hidden variables , 2006 .

[18]  Justin Dauwels,et al.  On Variational Message Passing on Factor Graphs , 2007, 2007 IEEE International Symposium on Information Theory.

[19]  Li Ping,et al.  The Factor Graph Approach to Model-Based Signal Processing , 2007, Proceedings of the IEEE.

[20]  H. B. Barlow,et al.  Unsupervised Learning , 1989, Neural Computation.