Introducing graphical models to analyze genetic programming dynamics

We propose graphical models as a new means of understanding genetic programming dynamics. Herein, we describe how to build an unbiased graphical model from a population of genetic programming trees. Graphical models both express information about the conditional dependency relations among a set of random variables and they support probabilistic inference regarding the likelihood of a random variable's outcome. We focus on the former information: by their structure, graphical models reveal structural dependencies between the nodes of genetic programming trees. We identify graphical model properties of potential interest in this regard - edge quantity and dependency among nodes expressed in terms of family relations. Using a simple symbolic regression problem we generate a graphical model of the population each generation. Then we interpret the graphical models with respect to conventional knowledge about the influence of subtree crossover and mutation upon tree structure.

[1]  Riccardo Poli,et al.  Foundations of Genetic Programming , 1999, Springer Berlin Heidelberg.

[2]  Hitoshi Iba,et al.  A Bayesian Network Approach to Program Generation , 2008, IEEE Transactions on Evolutionary Computation.

[3]  Paulien Hogeweg,et al.  Evolutionary Consequences of Coevolving Targets , 1997, Evolutionary Computation.

[4]  N. Hopper,et al.  Analysis of genetic diversity through population history , 1999 .

[5]  An Exploration of Structure Learning in Bayesian Networks , 2012 .

[6]  Kalyan Veeramachaneni,et al.  An investigation of local patterns for estimation of distribution genetic programming , 2012, GECCO '12.

[7]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[8]  Andrew W. Moore,et al.  Cached Sufficient Statistics for Efficient Machine Learning with Large Datasets , 1998, J. Artif. Intell. Res..

[9]  Jason M. Daida,et al.  Visualizing Tree Structures in Genetic Programming , 2003, GECCO.

[10]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[11]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine-mediated learning.

[12]  Leonardo Vanneschi,et al.  A Study of Fitness Distance Correlation as a Difficulty Measure in Genetic Programming , 2005, Evolutionary Computation.

[13]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[14]  Leonardo Vanneschi,et al.  Open issues in genetic programming , 2010, Genetic Programming and Evolvable Machines.

[15]  Moshe Sipper,et al.  Have your spaghetti and eat it too: evolutionary algorithmics and post-evolutionary analysis , 2010, Genetic Programming and Evolvable Machines.

[16]  Daphne Koller,et al.  Ordering-Based Search: A Simple and Effective Algorithm for Learning Bayesian Networks , 2005, UAI.

[17]  Nir Friedman,et al.  Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm , 1999, UAI.

[18]  Kalyan Veeramachaneni,et al.  Graphical models and what they reveal about GP when it solves a symbolic regression problem , 2012, GECCO '12.

[19]  Jason M. Daida,et al.  Phase Transitions in Genetic Programming Search , 2007 .

[20]  Günther R. Raidl,et al.  Empirical Analysis of Locality, Heritability and Heuristic Bias in Evolutionary Algorithms: A Case Study for the Multidimensional Knapsack Problem , 2005, Evolutionary Computation.

[21]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[22]  Arthur Carvalho,et al.  A cooperative coevolutionary genetic algorithm for learning bayesian network structures , 2011, GECCO '11.

[23]  C. D. MacLean,et al.  Chapter 2 A POPULATION BASED STUDY OF EVOLUTIONARY DYNAMICS IN GENETIC PROGRAMMING , .