Belief propagation and learning in convolution multi-layer factor graphs

In modeling time series, convolution multi-layer graphs are able to capture long-term dependence at a gradually increasing scale. We present an approach to learn a layered factor graph architecture starting from a stationary latent models for each layer. Simulations of belief propagation are reported for a three-layer graph on a small data set of characters.

[1]  Francesco Palmieri,et al.  Learning Non-Linear Functions With Factor Graphs , 2013, IEEE Transactions on Signal Processing.

[2]  Michael I. Jordan.  Graphical Models , 1998 .

[3]  Vincent Y. F. Tan,et al.  Learning Latent Tree Graphical Models , 2010, J. Mach. Learn. Res..

[4]  Francesco Palmieri,et al.  A Comparison of Algorithms for Learning Hidden Variables in Normal Graphs , 2013, ArXiv.

[5]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[6]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).

[7]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[8]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[9]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[10]  Marina Fruehauf,et al.  Nonlinear Programming Analysis And Methods , 2016 .

[11]  Frank R. Kschischang,et al.  On factor graphs and the Fourier transform , 2005, IEEE Transactions on Information Theory.

[12]  P. Dooren,et al.  Non-negative matrix factorization with fixed row and column sums , 2008 .

[13]  Francesco Palmieri,et al.  Simulink Implementation of Belief Propagation in Normal Factor Graphs , 2015, Advances in Neural Networks.

[14]  Y-Lan Boureau,et al.  Learning Convolutional Feature Hierarchies for Visual Recognition , 2010, NIPS.