Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms

In this paper we consider the optimal decomposition of Bayesian networks. More concretely, we examine empirically the applicability of genetic algorithms to the problem of the triangulation of moral graphs. This problem constitutes the only difficult step in the evidence propagation algorithm of Lauritzen and Spiegelhalter (1988) and is known to be NP-hard (Wen, 1991). We carry out experiments with distinct crossover and mutation operators and with different population sizes, mutation rates and selection biasses. The results are analysed statistically. They turn out to improve the results obtained with most other known triangulation methods (Kjærulff, 1990) and are comparable to results obtained with simulated annealing (Kjærulff, 1990; Kjærulff, 1992).

[1]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[2]  Pedro Larrañaga,et al.  Genetic Algorithms Elitist Probabilistic of Degree 1, a generalization of Simulated Annealing , 1993, AI*IA.

[3]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[4]  Judea Pearl,et al.  Evidential Reasoning Using Stochastic Simulation of Causal Models , 1987, Artif. Intell..

[5]  B. R. Fox,et al.  Genetic Operators for Sequencing Problems , 1990, FOGA.

[6]  D. J. Smith,et al.  A Study of Permutation Crossover Operators on the Traveling Salesman Problem , 1987, ICGA.

[7]  David E. Goldberg,et al.  Alleles, loci and the traveling salesman problem , 1985 .

[8]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[9]  G. Syswerda,et al.  Schedule Optimization Using Genetic Algorithms , 1991 .

[10]  Tomas Hrycej,et al.  Gibbs Sampling in Bayesian Networks , 1990, Artif. Intell..

[11]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[12]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[13]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[14]  Gregory F. Cooper,et al.  An Empirical Analysis of Likelihood-Weighting Simulation on a Large, Multiply-Connected Belief Network , 1991, Computers and biomedical research, an international journal.

[15]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[16]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[17]  Uffe Kjærulff Optimal decomposition of probabilistic networks by simulated annealing , 1992 .

[18]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[19]  Eric Horvitz,et al.  A Bayesian analysis of simulation algorithms for inference in belief networks , 1993, Networks.

[20]  Wilson X. Wen,et al.  Optimal decomposition of belief networks , 1990, UAI.

[21]  Lawrence Davis,et al.  Applying Adaptive Algorithms to Epistatic Domains , 1985, IJCAI.

[22]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[23]  David E. Goldberg,et al.  AllelesLociand the Traveling Salesman Problem , 1985, ICGA.

[24]  Heinz Mühlenbein,et al.  Parallel Genetic Algorithms, Population Genetics, and Combinatorial Optimization , 1989, Parallelism, Learning, Evolution.

[25]  Dirk Van Gucht,et al.  The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem , 1989 .

[26]  L. Darrell Whitley,et al.  Scheduling Problems and Traveling Salesmen: The Genetic Edge Recombination Operator , 1989, International Conference on Genetic Algorithms.

[27]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[28]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Hybrid Genetic Algorithms , 1995, AISTATS.

[29]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[30]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Dirk Van Gucht,et al.  Incorporating Heuristic Information into Genetic Search , 1987, International Conference on Genetic Algorithms.

[32]  Robert E. Tarjan,et al.  Algorithmic Aspects of Vertex Elimination on Graphs , 1976, SIAM J. Comput..

[33]  Darrell Whitley,et al.  The Travelling Salesman and Sequence Scheduling: Quality Solutions using Genetic Edge Recombination , 1990 .

[34]  Gregory F. Cooper,et al.  A randomized approximation algorithm for probabilistic inference on bayesian belief networks , 1990, Networks.

[35]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[36]  L. Darrell Whitley,et al.  Optimizing Neural Networks Using FasterMore Accurate Genetic Search , 1989, ICGA.

[37]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[38]  박철훈,et al.  Genetic Algorithm를 이용한 Traveling Salesman Problem 해법 , 1992 .

[39]  A. E. Eiben,et al.  Global Convergence of Genetic Algorithms: A Markov Chain Analysis , 1990, PPSN.

[40]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[41]  Ross D. Shachter Probabilistic Inference and Influence Diagrams , 1988, Oper. Res..

[42]  Robert E. Tarjan,et al.  Simple Linear-Time Algorithms to Test Chordality of Graphs, Test Acyclicity of Hypergraphs, and Selectively Reduce Acyclic Hypergraphs , 1984, SIAM J. Comput..

[43]  John J. Grefenstette,et al.  Genetic Algorithms for the Traveling Salesman Problem , 1985, ICGA.

[44]  Dirk Van Gucht,et al.  The Effects of Population SizeHeuristic Crossover and Local Improvement on a Genetic Algorithm for the Traveling Salesman Problem , 1989, ICGA.

[45]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[46]  Ross D. Shachter,et al.  Simulation Approaches to General Probabilistic Inference on Belief Networks , 2013, UAI.

[47]  Uffe Kjærulff,et al.  Blocking Gibbs sampling in very large probabilistic expert systems , 1995, Int. J. Hum. Comput. Stud..

[48]  Kuo-Chu Chang,et al.  Weighing and Integrating Evidence for Stochastic Simulation in Bayesian Networks , 2013, UAI.

[49]  Uue Kjjrull Triangulation of Graphs { Algorithms Giving Small Total State Space Triangulation of Graphs { Algorithms Giving Small Total State Space , 1990 .

[50]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[51]  L. D. Whitley,et al.  The Traveling Salesman and Sequence Scheduling : , 1990 .

[52]  Uday Kumar Chakraborty,et al.  Using Reliability Analysis to Estimate the Number of Generations to Convergence in Genetic Algorithms , 1993, Inf. Process. Lett..

[53]  Gunar E. Liepins,et al.  A New Approach on the Traveling Salesman Problem by Genetic Algorithms , 1993, ICGA.

[54]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .