Discrete independent component analysis (DICA) with belief propagation

We apply belief propagation to a Bayesian bipartite graph composed of discrete independent hidden variables and discrete visible variables. The network is the Discrete counterpart of Independent Component Analysis (DICA) and it is manipulated in a factor graph form for inference and learning. A full set of simulations is reported for character images from the MNIST dataset. The results show that the factorial code implemented by the sources contributes to build a good generative model for the data that can be used in various inference modes.

[1]  Francesco Palmieri,et al.  A Comparison of Algorithms for Learning Hidden Variables in Normal Graphs , 2013, ArXiv.

[2]  Francesco Palmieri,et al.  Simulink Implementation of Belief Propagation in Normal Factor Graphs , 2015, Advances in Neural Networks.

[3]  Francesco Palmieri,et al.  Learning Non-Linear Functions With Factor Graphs , 2013, IEEE Transactions on Signal Processing.

[4]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Processing Magazine.

[5]  Aapo Hyvärinen,et al.  Statistical Models of Natural Images and Cortical Visual Representation , 2010, Top. Cogn. Sci..

[6]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[7]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[8]  M. Castella,et al.  Inversion of Polynomial Systems and Separation of Nonlinear Mixtures of Finite-Alphabet Sources , 2008, IEEE Transactions on Signal Processing.

[9]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[10]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[11]  Michael I. Jordan.  Graphical Models , 1998 .

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  Arie Yeredor,et al.  ICA over finite fields - Separability and algorithms , 2012, Signal Process..

[14]  Aleks Jakulin,et al.  Discrete Component Analysis , 2005, SLSFS.

[15]  Aapo Hyvärinen,et al.  Natural Image Statistics - A Probabilistic Approach to Early Computational Vision , 2009, Computational Imaging and Vision.

[16]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[17]  Yannick Deville,et al.  A Quantum-feedforward and classical-feedback separating structure adapted with monodirectional measurements; Blind qubit uncoupling capability and links with ICA , 2013, 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).