Estimation of Bayesian networks algorithms in a class of complex networks

In many optimization problems, regardless of the domain to which it belongs, the structural component that the interactions among variables provides can be seen as a network. The impact that the topological characteristics of that network has, both in the hardness of the problem and in the performance of the optimization techniques, constitutes a very important subject of research. In this paper, we study the behavior of estimation of distribution algorithms (EDAs) in functions whose structure is defined by using different network topologies which include grids, small-world networks and random graphs. In order to do that, we use several descriptors such as the population size, the number of evaluations as well as the structures learned during the search. Furthermore, we take measures from the field of complex networks such as clustering coefficient or characteristic path length in order to quantify the topological properties of the function structure and analyze their relation with the behavior of EDAs. The results show that these measures are useful to have better understanding of this type of algorithms which have exhibited a high sensitivity to the topological characteristics of the function structure. This study creates a link between EDAs based on Bayesian networks and the emergent field of complex networks.

[1]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[2]  L. Pauling,et al.  A Theory of Ferromagnetism. , 1953, Proceedings of the National Academy of Sciences of the United States of America.

[3]  Concha Bielza,et al.  Mining probabilistic models learned by EDAs in the optimization of multi-objective problems , 2009, GECCO.

[4]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[5]  David E. Goldberg,et al.  Hierarchical BOA Solves Ising Spin Glasses and MAXSAT , 2003, GECCO.

[6]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[7]  Th.W. Ruijgrok.  On the theory of ferromagnetism , 1962 .

[8]  E. Ising.  Beitrag zur Theorie des Ferromagnetismus , 1925 .

[9]  Pedro Larrañaga,et al.  Protein Folding in Simplified Models With Estimation of Distribution Algorithms , 2008, IEEE Transactions on Evolutionary Computation.

[10]  Concha Bielza,et al.  Mateda-2.0: A MATLAB package for the implementation and analysis of estimation of distribution algorithms , 2010 .

[11]  Martin V. Butz,et al.  Performance of evolutionary algorithms on NK landscapes with nearest neighbor interactions and tunable overlap , 2009, GECCO '09.

[12]  Concha Bielza,et al.  A review of estimation of distribution algorithms in bioinformatics , 2008, BioData Mining.

[13]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[14]  Yong Gao,et al.  Space Complexity of Estimation of Distribution Algorithms , 2005, Evolutionary Computation.

[15]  Ruhul A. Sarker,et al.  The Self-Organization of Interaction Networks for Nature-Inspired Optimization , 2008, IEEE Transactions on Evolutionary Computation.

[16]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[17]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[18]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[19]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[20]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[21]  Margaret J. Eppstein,et al.  Emergent mating topologies in spatially structured genetic algorithms , 2006, GECCO.

[22]  Paul Erdös,et al.  On random graphs, I , 1959 .

[23]  David E. Goldberg,et al.  Using Previous Models to Bias Structural Learning in the Hierarchical BOA , 2012, Evolutionary Computation.

[24]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[25]  José Antonio Lozano Alonso,et al.  A quantitative analysis of estimation of distribution algorithms based on Bayesian networks , 2009 .

[26]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[27]  Meng-Hiot Lim,et al.  Linkage in Evolutionary Computation , 2008, Linkage in Evolutionary Computation.

[28]  Massimo Marchiori,et al.  Error and attacktolerance of complex network s , 2004 .

[29]  Martin Pelikan,et al.  From mating pool distributions to model overfitting , 2008, GECCO '08.

[30]  J. A. Lozano,et al.  An empirical analysis of loopy belief propagation in three topologies : grids , small-world networks and random graphs , 2008 .

[31]  Roberto Santana,et al.  Analyzing the probability of the optimum in EDAs based on Bayesian networks , 2009, 2009 IEEE Congress on Evolutionary Computation.

[32]  S. Strogatz.  Exploring complex networks , 2001, Nature.

[33]  Endika Bengoetxea,et al.  Inexact Graph Matching Using Estimation of Distribution Algorithms , 2002 .

[34]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[35]  Sergey N. Dorogovtsev,et al.  Critical phenomena in complex networks , 2007, ArXiv.

[36]  Susan Khor,et al.  Effect of degree distribution on evolutionary search , 2009, GECCO.

[37]  Pedro Larrañaga,et al.  Exact Bayesian network learning in estimation of distribution algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[38]  Heinz Mühlenbein,et al.  The Estimation of Distributions and the Minimum Relative Entropy Principle , 2005, Evol. Comput..

[39]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[40]  Martin Pelikan,et al.  Analyzing Probabilistic Models in Hierarchical BOA , 2009, IEEE Transactions on Evolutionary Computation.