Learning Bayesian networks in the space of structures by estimation of distribution algorithms

The induction of the optimal Bayesian network structure is NP‐hard, justifying the use of search heuristics. Two novel population‐based stochastic search approaches, univariate marginal distribution algorithm (UMDA) and population‐based incremental learning (PBIL), are used to learn a Bayesian network structure from a database of cases in a score + search framework. A comparison with a genetic algorithm (GA) approach is performed using three different scores: penalized maximum likelihood, marginal likelihood, and information‐theory–based entropy. Experimental results show the interesting capabilities of both novel approaches with respect to the score value and the number of generations needed to converge. © 2003 Wiley Periodicals, Inc.

[1]  Pedro Larrañaga,et al.  An improved Bayesian structural EM algorithm for learning Bayesian networks for clustering , 2000, Pattern Recognit. Lett..

[2]  Jin Tian,et al.  A Branch-and-Bound Algorithm for MDL Learning Bayesian Networks , 2000, UAI.

[3]  R. W. Robinson.  Counting unlabeled acyclic digraphs , 1977 .

[4]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[5]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[7]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[8]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[9]  Michael D. Perlman,et al.  Enumerating Markov Equivalence Classes of Acyclic Digraph Models , 2001, UAI.

[10]  G. Bortolan,et al.  The problem of linguistic approximation in clinical decision making , 1988, Int. J. Approx. Reason..

[11]  Robert Castelo,et al.  Improved learning of Bayesian networks , 2001, UAI.

[12]  Pedro Larrañaga,et al.  Combinatonal Optimization by Learning and Simulation of Bayesian Networks , 2000, UAI.

[13]  J. A. Lozano,et al.  Analyzing the PBIL Algorithm by Means of Discrete Dynamical Systems , 2000 .

[14]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[15]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.

[16]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[17]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[18]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[19]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[20]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[21]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[22]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.

[23]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[24]  Harald Steck.  On the Use of Skeletons when Learning in Bayesian Networks , 2000, UAI.

[25]  Kwong-Sak Leung,et al.  Using Evolutionary Programming and Minimum Description Length Principle for Data Mining of Bayesian Networks , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.