Network measures for information extraction in evolutionary algorithms

Abstract Problem domain information extraction is a critical issue in many real-world optimization problems. Increasing the repertoire of techniques available in evolutionary algorithms with this purpose is fundamental for extending the applicability of these algorithms. In this paper we introduce a unifying information mining approach for evolutionary algorithms. Our proposal is based on a division of the stages where structural modelling of the variables interactions is applied. Particular topological characteristics induced from different stages of the modelling process are identified. Network theory is used to harvest problem structural information from the learned probabilistic graphical models (PGMs). We show how different statistical measures, previously studied for networks from different domains, can be applied to mine the graphical component of PGMs. We provide evidence that the computed measures can be employed for studying problem difficulty, classifying different problem instances and predict...

[1]  Martin Pelikan,et al.  From mating pool distributions to model overfitting , 2008, GECCO '08.

[2]  Edmund K. Burke,et al.  Multimeme Algorithms for Protein Structure Prediction , 2002, PPSN.

[3]  D. Watts,et al.  Small Worlds: The Dynamics of Networks between Order and Randomness , 2001 .

[4]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[5]  W. Marsden.  I and J , 2012 .

[6]  Pedro Larrañaga,et al.  Side chain placement using estimation of distribution algorithms , 2007, Artif. Intell. Medicine.

[7]  Zeynep F. Altun,et al.  Neuroscience Databases , 2003, Springer US.

[8]  Shumeet Baluja,et al.  Incorporating a priori Knowledge in Probabilistic-Model Based Optimization , 2006, Scalable Optimization via Probabilistic Modeling.

[9]  Concha Bielza,et al.  Multi-objective Optimization with Joint Probabilistic Modeling of Objectives and Variables , 2011, EMO.

[10]  Roberto Santana,et al.  Analyzing the probability of the optimum in EDAs based on Bayesian networks , 2009, 2009 IEEE Congress on Evolutionary Computation.

[11]  E A Leicht,et al.  Community structure in directed networks. , 2007, Physical review letters.

[12]  Falk Schreiber,et al.  Analysis of Biological Networks , 2008 .

[13]  Roberto Santana,et al.  Toward Understanding EDAs Based on Bayesian Networks Through a Quantitative Analysis , 2012, IEEE Transactions on Evolutionary Computation.

[14]  H. Berendse,et al.  The application of graph theoretical analysis to complex networks in the brain , 2007, Clinical Neurophysiology.

[15]  Roberto Santana,et al.  Estimation of Distribution Algorithms with Kikuchi Approximations , 2005, Evolutionary Computation.

[16]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[17]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[18]  D. Böhning.  Multinomial logistic regression algorithm , 1992 .

[19]  Heinz Mühlenbein,et al.  Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning , 2002, Int. J. Approx. Reason..

[20]  Julio M. Ottino,et al.  Complex networks , 2004, Encyclopedia of Big Data.

[21]  Carlos A. Coello Coello,et al.  Objective reduction using a feature selection technique , 2008, GECCO '08.

[22]  Martin Pelikan,et al.  Scalable Optimization via Probabilistic Modeling , 2006, Studies in Computational Intelligence.

[23]  Qingfu Zhang,et al.  Approaches to selection and their effect on fitness modelling in an Estimation of Distribution Algorithm , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[24]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[25]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[26]  Lenny Moss,et al.  What genes can't do , 2002 .

[27]  Olaf Sporns,et al.  Complex network measures of brain connectivity: Uses and interpretations , 2010, NeuroImage.

[28]  Jie Wu,et al.  Small Worlds: The Dynamics of Networks between Order and Randomness , 2003 .

[29]  Carlos Cotta,et al.  Protein Structure Prediction Using Evolutionary Algorithms Hybridized with Backtracking , 2009, IWANN.

[30]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[31]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[32]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[33]  U. Brandes.  A faster algorithm for betweenness centrality , 2001 .

[34]  D. Long.  Networks of the Brain , 2011 .

[35]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[36]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[37]  O. Sporns,et al.  Motifs in Brain Networks , 2004, PLoS biology.

[38]  A. Vespignani,et al.  The architecture of complex weighted networks. , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[39]  Andreas Zell,et al.  Inferring Disease-Related Metabolite Dependencies with a Bayesian Optimization Algorithm , 2012, EvoBIO.

[40]  José Antonio Lozano Alonso,et al.  A quantitative analysis of estimation of distribution algorithms based on Bayesian networks , 2009 .

[41]  Shiva Kintali,et al.  Betweenness Centrality : Algorithms and Lower Bounds , 2008, ArXiv.

[42]  Stuart A. Kauffman,et al.  ORIGINS OF ORDER , 2019, Origins of Order.

[43]  David E. Goldberg,et al.  Influence of selection and replacement strategies on linkage learning in BOA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[44]  Concha Bielza,et al.  A review on probabilistic graphical models in evolutionary computation , 2012, J. Heuristics.

[45]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[46]  T. Hastie,et al.  Classification of gene microarrays by penalized logistic regression. , 2004, Biostatistics.

[47]  Isabelle Bloch,et al.  Inexact graph matching by means of estimation of distribution algorithms , 2002, Pattern Recognit..

[48]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[49]  Anil K. Jain,et al.  On the optimal number of features in the classification of multivariate Gaussian data , 1978, Pattern Recognit..

[50]  David E. Goldberg,et al.  Using Previous Models to Bias Structural Learning in the Hierarchical BOA , 2012, Evolutionary Computation.

[51]  R. Tibshirani,et al.  Classification and prediction of clinical Alzheimer's diagnosis based on plasma signaling proteins , 2007, Nature Medicine.

[52]  Pedro Larrañaga,et al.  Feature Subset Selection by Bayesian network-based optimization , 2000, Artif. Intell..

[53]  V. Latora,et al.  Complex networks: Structure and dynamics , 2006 .

[54]  Kate Smith-Miles,et al.  Cross-disciplinary perspectives on meta-learning for algorithm selection , 2009, CSUR.

[55]  Martin Pelikan.  Analysis of estimation of distribution algorithms and genetic algorithms on NK landscapes , 2008, GECCO '08.

[56]  L. Amaral,et al.  Small-World Networks: Evidence for a Crossover Picture , 1999, cond-mat/9903108.

[57]  Roberto Santana,et al.  Structural transfer using EDAs: An application to multi-marker tagging SNP selection , 2012, 2012 IEEE Congress on Evolutionary Computation.

[58]  Pedro Larrañaga,et al.  Protein Folding in Simplified Models With Estimation of Distribution Algorithms , 2008, IEEE Transactions on Evolutionary Computation.

[59]  S. Shen-Orr,et al.  Network motifs: simple building blocks of complex networks. , 2002, Science.

[60]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[61]  Concha Bielza,et al.  Regularized continuous estimation of distribution algorithms , 2013, Appl. Soft Comput..

[62]  T. Vicsek,et al.  Uncovering the overlapping community structure of complex networks in nature and society , 2005, Nature.

[63]  Siddhartha Shakya,et al.  Using a Markov network model in a univariate EDA: an empirical cost-benefit analysis , 2005, GECCO '05.

[64]  K. Dill.  Theory for the folding and stability of globular proteins. , 1985, Biochemistry.

[65]  Martin Pelikan,et al.  Fitness Inheritance in the Bayesian Optimization Algorithm , 2004, GECCO.

[66]  M. E. J. Newman,et al.  Mixing patterns in networks: Empirical results and models , 2002 .

[67]  Pedro Larrañaga,et al.  The Role of a Priori Information in the Minimization of Contact Potentials by Means of Estimation of Distribution Algorithms , 2007, EvoBIO.

[68]  Concha Bielza,et al.  Mateda-2.0: A MATLAB package for the implementation and analysis of estimation of distribution algorithms , 2010 .

[69]  Siddhartha Shakya,et al.  Optimization by estimation of distribution with DEUM framework based on Markov random fields , 2007, Int. J. Autom. Comput..

[70]  Sergey N. Dorogovtsev,et al.  Critical phenomena in complex networks , 2007, ArXiv.

[71]  Pedro Larrañaga,et al.  Exact Bayesian network learning in estimation of distribution algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[72]  U. Alon,et al.  Spontaneous evolution of modularity and network motifs. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[73]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[74]  Yong Gao,et al.  Space Complexity of Estimation of Distribution Algorithms , 2005, Evolutionary Computation.

[75]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[76]  Pedro Larrañaga,et al.  The Impact of Exact Probabilistic Learning Algorithms in EDAs Based on Bayesian Networks , 2008, Linkage in Evolutionary Computation.

[77]  H E Stanley,et al.  Classes of small-world networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[78]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[79]  Concha Bielza,et al.  Peakbin Selection in Mass Spectrometry Data Using a Consensus Approach with Estimation of Distribution Algorithms , 2011, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[80]  Heinz Mühlenbein,et al.  The Estimation of Distributions and the Minimum Relative Entropy Principle , 2005, Evol. Comput..

[81]  Martin Pelikan,et al.  Transfer Learning, Soft Distance-Based Bias, and the Hierarchical BOA , 2012, PPSN.

[82]  Concha Bielza,et al.  Optimizing Brain Networks Topologies Using Multi-objective Evolutionary Computation , 2011, Neuroinformatics.

[83]  Martin Pelikan,et al.  Enhancing Efficiency of Hierarchical BOA Via Distance-Based Model Restrictions , 2008, PPSN.

[84]  Pedro Larrañaga,et al.  Towards a New Evolutionary Computation - Advances in the Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[85]  R. Guimerà,et al.  Functional cartography of complex metabolic networks , 2005, Nature.

[86]  Concha Bielza,et al.  Mining probabilistic models learned by EDAs in the optimization of multi-objective problems , 2009, GECCO.

[87]  J. Hirst,et al.  The evolutionary landscape of functional model proteins. , 1999, Protein engineering.

[88]  Olaf Sporns,et al.  Graph Theory Methods for the Analysis of Neural Connectivity Patterns , 2003 .

[89]  Aldons J. Lusis,et al.  What Genes Can't Do , 2003, Nature Medicine.

[90]  David E. Goldberg,et al.  Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).