COMPUTATIONAL OPTIMIZATION FOR NORMAL FORM REALIZATION OF BAYESIAN MODEL GRAPHS

Bayesian networks in their Factor Graph Reduced Normal Form (FGrn) represent a very appealing paradigm for the realization of structures for probabilistic inference. Unfortunately, the computational and memory complexity of such networks remains high, especially if the network has to extend to large structures such as multi-layers and highly connected graphs. In this paper we focus on the details on probability propagation and learning that can reduce such complexity. More specifically we propose new algorithms and proceed to create a library that allows a significant reduction in costs with respect to direct use of the standard sums-products and Maximum Likelihood (ML) learning. Analysis and results are presented with reference to a Latent Variable Model (LVM).

[1]  H.-A. Loeliger,et al.  An introduction to factor graphs , 2004, IEEE Signal Processing Magazine.

[2]  Francesco Palmieri,et al.  Discrete independent component analysis (DICA) with belief propagation , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[3]  Francesco Palmieri,et al.  A Comparison of Algorithms for Learning Hidden Variables in Bayesian Factor Graphs in Reduced Normal Form , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[4]  M. Elter,et al.  The prediction of breast cancer biopsy outcomes using two CAD approaches that both emphasize an intelligible decision process. , 2007, Medical physics.

[5]  David Barber,et al.  Bayesian reasoning and machine learning , 2012 .

[6]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[7]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[8]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[9]  Christopher M. Bishop.  Latent Variable Models , 1998, Learning in Graphical Models.

[10]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[11]  Francesco Palmieri,et al.  Two-dimensional multi-layer Factor Graphs in Reduced Normal Form , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[12]  G. Forney,et al.  Codes on graphs: normal realizations , 2000, 2000 IEEE International Symposium on Information Theory (Cat. No.00CH37060).