An Introduction to Probabilistic Graphical Models

In this chapter we will introduce two probabilistic graphical models -Bayesian networks and Gaussian networks-that will be used to carry out factorization of the probability distribution of the selected individuals in the Estimation of Distribution Algorithms based approaches. For both paradigms we will present different algorithms to induce the underlying model from data, as well as some methods to simulate such models.

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  J McCarthy,et al.  MEASURES OF THE VALUE OF INFORMATION. , 1956, Proceedings of the National Academy of Sciences of the United States of America.

[3]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[4]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[5]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[6]  H. Akaike.  A new look at the statistical model identification , 1974 .

[7]  N. Wermuth.  Model Search among Multiplicative Models , 1976 .

[8]  George Marsaglia,et al.  Improvements on Fast Methods for Generating Normal Random Variables , 1976, Inf. Process. Lett..

[9]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[10]  D. J. Best,et al.  Some Easily Programmed Pseudo-Random Normal Generators , 1979, Aust. Comput. J..

[11]  A. Dawid.  Conditional Independence in Statistical Theory , 1979 .

[12]  B. Ripley.  Computer generation of random variables: a tutorial , 1983 .

[13]  Yoshiko Wakabayashi,et al.  On k-leaf-connected graphs , 1986, Journal of combinatorial theory. Series B (Print).

[14]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[15]  T. Speed,et al.  Gaussian Markov Distributions over Finite Graphs , 1986 .

[16]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[17]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  Judea Pearl,et al.  Evidential Reasoning Using Stochastic Simulation of Causal Models , 1987, Artif. Intell..

[19]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[20]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[21]  Ronald A. Howard,et al.  Readings on the Principles and Applications of Decision Analysis , 1989 .

[22]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[23]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[24]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[25]  Kristian G. Olesen,et al.  An algebra of bayesian belief universes for knowledge-based systems , 1990, Networks.

[26]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.

[27]  Tomas Hrycej,et al.  Gibbs Sampling in Bayesian Networks , 1990, Artif. Intell..

[28]  J. N. R. Jeffers,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[29]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems , 1990 .

[30]  Gregory F. Cooper,et al.  A randomized approximation algorithm for probabilistic inference on bayesian belief networks , 1990, Networks.

[31]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[32]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[33]  Eric Horvitz,et al.  Decision Analysis and Expert Systems , 1991, AI Mag..

[34]  Eugene Charniak,et al.  Bayesian Networks without Tears , 1991, AI Mag..

[35]  Gregory F. Cooper,et al.  An Empirical Analysis of Likelihood-Weighting Simulation on a Large, Multiply-Connected Belief Network , 1991, Computers and biomedical research, an international journal.

[36]  P. Spirtes,et al.  An Algorithm for Fast Recovery of Sparse Causal Graphs , 1991 .

[37]  Judea Pearl,et al.  Belief Networks Revisited , 1993, Artif. Intell..

[38]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[39]  Eric Horvitz,et al.  A Bayesian analysis of simulation algorithms for inference in belief networks , 1993, Networks.

[40]  David Heckerman,et al.  Learning Gaussian Networks , 1994, UAI.

[41]  D. Madigan,et al.  Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window , 1994 .

[42]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[43]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms , 1994 .

[44]  David Heckerman,et al.  Learning Bayesian Networks: Search Methods and Experimental Results , 1995 .

[45]  Michael P. Wellman,et al.  Bayesian networks , 1995, CACM.

[46]  R. Bouckaert.  Bayesian belief networks : from construction to inference , 1995 .

[47]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[48]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[49]  Enrique F. Castillo,et al.  A modified simulation scheme for inference in Bayesian networks , 1996, Int. J. Approx. Reason..

[50]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Glenn Shafer,et al.  Probabilistic expert systems , 1996, CBMS-NSF regional conference series in applied mathematics.

[52]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[53]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[54]  Ulises Cortés,et al.  Learning Causal Networks from Data: A Survey and a New Algorithm for Recovering Possibilistic Causal Networks , 1997, AI Commun..

[55]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[56]  Joe Whittaker,et al.  Edge Exclusion Tests for Graphical Gaussian Models , 1999, Learning in Graphical Models.

[57]  Paul J. Krause,et al.  Learning probabilistic networks , 1999, The Knowledge Engineering Review.

[58]  Kwong-Sak Leung,et al.  Using Evolutionary Programming and Minimum Description Length Principle for Data Mining of Bayesian Networks , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[60]  Kathryn B. Laskey,et al.  Learning Bayesian Networks from Incomplete Data with Stochastic Search Algorithms , 1999, UAI.

[61]  A. Salmerón,et al.  Importance sampling in Bayesian networks using probability trees , 2000 .

[62]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[63]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[64]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.