A Bayesian method for the induction of probabilistic networks from data

This paper presents a Bayesian method for constructing probabilistic networks from databases. In particular, we focus on constructing Bayesian belief networks. Potential applications include computer-assisted hypothesis testing, automated scientific discovery, and automated construction of probabilistic expert systems. We extend the basic method to handle missing data and hidden (latent) variables. We show how to perform probabilistic inference by averaging over the inferences of multiple belief networks. Results are presented of a preliminary evaluation of an algorithm for constructing a belief network from a database of cases. Finally, we relate the methods in this paper to previous work, and we discuss open problems.

[1]  R. F.,et al.  Mathematical Statistics , 1944, Nature.

[2]  G. Enderlein.  Wilks, S. S.: Mathematical Statistics. J. Wiley and Sons, New York–London 1962; 644 S., 98 s , 1964 .

[3]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[4]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[5]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[6]  David Lindley,et al.  Optimal Statistical Decisions , 1971 .

[7]  R. W. Robinson.  Counting unlabeled acyclic digraphs , 1977 .

[8]  Walter Brötz,et al.  Electronically produced equidensities from time exposures and instantaneous photographs in the investigation of pool flames , 1980 .

[9]  R L Blum,et al.  Discovery, confirmation, and incorporation of causal relationships from a large time-oriented clinical data base: the RX project. , 1982, Computers and biomedical research, an international journal.

[10]  N. Wermuth,et al.  Graphical and recursive models for contingency tables , 1983 .

[11]  Peter C. Cheeseman,et al.  A Method of Computing Generalized Bayesian Probability Values for Expert Systems , 1983, IJCAI.

[12]  Gregory F. Cooper,et al.  NESTOR: A Computer-Based Medical Diagnostic Aid That Integrates Causal and Probabilistic Knowledge. , 1984 .

[13]  George J. Klir,et al.  Reconstructability Analysis: An Overview , 1984 .

[14]  T. Speed,et al.  Recursive causal models , 1984, Journal of the Australian Mathematical Society. Series A. Pure Mathematics and Statistics.

[15]  John R. Anderson,et al.  Machine learning - an artificial intelligence approach , 1982, Symbolic computation.

[16]  Ross D. Shachter.  Intelligent Probabilistic Inference , 1985, UAI.

[17]  P. Ut Goff,et al.  Machine learning of inductive bias , 1986 .

[18]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[19]  Steen Andreassen,et al.  MUNIN - A Causal Probabilistic Network for Interpretation of Electromyographic Findings , 1987, IJCAI.

[20]  Max Henrion,et al.  An Experimental Comparison of Knowledge Engineering for Expert Systems and for Decision Analysis , 1987, AAAI.

[21]  Alice M. Agogino,et al.  IDES: influence diagram based expert system , 1987 .

[22]  Nasa Ames Reseach.  AUTOMATIC PROBABILISTIC KNOWLEDGE ACQUISITION FROM DATA , 1987 .

[23]  William B. Gevarter,et al.  Automatic probabilistic knowledge acquisition from data , 1987, 1987 IEEE Third International Conference on Data Engineering.

[24]  Judea Pearl,et al.  The recovery of causal poly-trees from statistical data , 1987, Int. J. Approx. Reason..

[25]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[26]  Ross D. Shachter.  Probabilistic Inference and Influence Diagrams , 1988, Oper. Res..

[27]  Gregory F. Cooper,et al.  KNET: integrating hypermedia and normative bayesian modeling , 1988, UAI.

[28]  Ronald A. Howard.  Uncertainty about Probability: A Decision Analysis Perspective , 1988 .

[29]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[30]  Eric Horvitz,et al.  Decision theory in expert systems and artificial intelligenc , 1988, Int. J. Approx. Reason..

[31]  Ross D. Shachter.  A linear approximation method for probabilistic inference , 2013, UAI.

[32]  Samuel Holtzman,et al.  Intelligent decision systems , 1988 .

[33]  Chris Carter,et al.  Multiple decision trees , 2013, UAI.

[34]  Matthew Self,et al.  Bayesian Classification , 1988, AAAI.

[35]  P. Spirtes,et al.  Latent variables, causal models and overidentifying constraints , 1988 .

[36]  Alice M. Agogino,et al.  Automated Construction of Sparse Bayesian Networks from Unstructured Probabilistic Models and Domain Information , 2013, UAI.

[37]  Gregory F. Cooper,et al.  Response to the discussion of the paper ‘current research directions in the development of expert systems based on belief networks’ , 1989 .

[38]  Kevin T. Kelly,et al.  Discovering Causal Structure. , 1989 .

[39]  Max Henrion,et al.  An Introduction to Algorithms for Inference in Belief Nets , 1989, UAI.

[40]  Gregory F. Cooper,et al.  Current research directions in the development of expert systems based on belief networks , 1989 .

[41]  Douglas W. Nychka,et al.  Discovering Causal Structure , 1989 .

[42]  H. J. Suermondt,et al.  Probabilistic Prediction of the Outcome of Bone-Marrow Transplantation , 1989 .

[43]  Geoffrey E. Hinton.  Connectionist Learning Procedures , 1989, Artif. Intell..

[44]  Jaime G. Carbonell,et al.  Introduction: Paradigms for Machine Learning , 1989, Artif. Intell..

[45]  P. Spirtes,et al.  Causality From Probability , 1989 .

[46]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[47]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[48]  Stuart L. Crawford,et al.  An architecture for probabilistic concept-based information retrieval , 1989, SIGIR '90.

[49]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[50]  Richard Scheines,et al.  Simulation Studies of the Reliability of Computer-Aided Model Specification Using the TETRAD II, EQS, and LISREL Programs , 1990 .

[51]  Dan Geiger,et al.  Learning Causal Trees from Dependence Information , 1990, AAAI.

[52]  P. Spirtes,et al.  Causal structure among measured variables preserved with unmeasured variables , 1990 .

[53]  Wray L. Buntine.  Myths and Legends in Learning Classification Rules , 1990, AAAI.

[54]  David Heckerman,et al.  Probabilistic similarity networks , 1991, Networks.

[55]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems , 1990 .

[56]  David J. Spiegelhalter,et al.  Sequential updating of conditional probabilities on directed graphical structures , 1990, Networks.

[57]  Stuart L. Crawford,et al.  Constructor: A System for the Induction of Probabilistic Models , 1990, AAAI.

[58]  Wray L. Buntine,et al.  A theory of learning classification rules , 1990 .

[59]  David Heckerman,et al.  Advances in Probabilistic Reasoning , 1994, Conference on Uncertainty in Artificial Intelligence.

[60]  P. Spirtes,et al.  An Algorithm for Fast Recovery of Sparse Causal Graphs , 1991 .

[61]  A. Mallet,et al.  Learning probabilities in causal trees from incomplete databases , 1991 .

[62]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[63]  Edward H. Herskovits,et al.  Computer-based probabilistic-network construction , 1992 .

[64]  Stuart L. Crawford,et al.  An analysis of two probabilistic model induction techniques , 1992 .

[65]  Ivan Bratko,et al.  Machine learning in artificial intelligence , 1993, Artif. Intell. Eng..

[66]  Shinichi Morishita,et al.  On Classification and Regression , 1998, Discovery Science.

[67]  Kathryn B. Laskey,et al.  Uncertainty in Artificial Intelligence 15 , 1999 .

[68]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[69]  Charles E. Heckler,et al.  Applied Multivariate Statistical Analysis , 2005, Technometrics.