Learning Bayesian Networks In The Space Of Orderings With Estimation Of Distribution Algorithms

The search for the optimal ordering of a set of variables in order to solve a computational problem is a dicult y that can appear in several circumstances. One of these situations is the automatic learning of a network structure, for example, a Bayesian Network structure (BN) starting from a dataset. Searching in the space of structures is often unmanageable, especially if the number of variables is high. Popular heuristic approaches, like Cooper and Herskovits’s K2 algorithm, depend on a given ordering of variables. Estimation of Distribution Algorithms (EDAs) are a new paradigm for Evolutionary Computation that have been used as a search engine in the BN structure learning problem. In this paper, we will use two dieren t EDAs to obtain not the best structure, but the optimal ordering of variables for the K2 algorithm: UMDA and MIMIC, both of them in discrete and continuous domains. We will also check whether the individual representation and its relation to the corresponding ordering play important roles, and whether MIMIC outperforms the results of UMDA.

[1]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[2]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[3]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[4]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[5]  Daniel Kahneman,et al.  Probabilistic reasoning , 1993 .

[6]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[7]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[8]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[9]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[10]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[13]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[14]  Luis M. de Campos,et al.  The Search of Causal Orderings: A Short Cut for Learning Belief Networks , 2001, ECSQARU.

[15]  C. Robert Kenley,et al.  Gaussian influence diagrams , 1989 .

[16]  Luis M. de Campos,et al.  An Algorithm for Finding Minimum d-Separating Sets in Belief Networks , 1996, UAI.

[17]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[18]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[19]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[20]  Nenad Mladenović,et al.  An Introduction to Variable Neighborhood Search , 1997 .

[21]  Thomas Dyhre Nielsen,et al.  Symbolic and Quantitative Approaches to Reasoning with Uncertainty , 2003, Lecture Notes in Computer Science.

[22]  Pedro Larrañaga,et al.  Learning Bayesian networks in the space of structures by estimation of distribution algorithms , 2003, Int. J. Intell. Syst..

[23]  Isabelle Bloch,et al.  Inexact graph matching by means of estimation of distribution algorithms , 2002, Pattern Recognit..

[24]  Remco R. Bouckaert,et al.  Optimizing Causal Orderings for Generating DAGs from Data , 1992, UAI.

[25]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[26]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[27]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[28]  Pedro Larrañaga,et al.  Unsupervised Learning Of Bayesian Networks Via Estimation Of Distribution Algorithms: An Application To Gene Expression Data Clustering , 2004, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[29]  H. Mühlenbein,et al.  Gene Pool Recombination in Genetic Algorithms , 1996 .

[30]  Jose Miguel Puerta,et al.  Stochastic Local Algorithms for Learning Belief Networks: Searching in the Space of the Orderings , 2001, ECSQARU.