On the complexity of hierarchical problem solving

Competent Genetic Algorithms can efficiently address problems in which the linkage between variables is limited to a small order k. Problems with higher order dependencies can only be addressed efficiently if further problem properties exist that can be exploited. An important class of problems for which this occurs is that of hierarchical problems. Hierarchical problems can contain dependencies between all variables (k=n) while being solvable in polynomial time.An open question so far is what precise properties a hierarchical problem must possess in order to be solvable efficiently. We study this question by investigating several features of hierarchical problems and determining their effect on computational complexity, both analytically and empirically. The analyses are based on the Hierarchical Genetic Algorithm (HGA), which is developed as part of this work. The HGA is tested on ranges of hierarchical problems, produced by a generator for hierarchical problems.

[1]  David E. Goldberg,et al.  A hierarchy machine: Learning to optimize from nature and humans , 2003, Complex..

[2]  Nir Friedman,et al.  On the Sample Complexity of Learning Bayesian Networks , 1996, UAI.

[3]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[4]  J. Pollack,et al.  Compositional evolution: interdisciplinary investigations in evolvability, modularity, and symbiosis , 2002 .

[5]  G. Harik Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[6]  Kalyanmoy Deb,et al.  RapidAccurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms , 1993, ICGA.

[7]  Dirk Thierens,et al.  Hierarchical Genetic Algorithms , 2004, PPSN.

[8]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[9]  H. Kargupta SEARCH , Evolution , And The Gene Expression Messy Genetic Algorithm , 1994 .

[10]  John R. Koza Genetic Programming III - Darwinian Invention and Problem Solving , 1999, Evolutionary Computation.

[11]  J. Pollack,et al.  A computational model of symbiotic composition in evolutionary transitions. , 2003, Bio Systems.

[12]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[13]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[14]  Marc Toussaint,et al.  Compact Genetic Codes as a Search Strategy of Evolutionary Processes , 2005, FOGA.

[15]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[16]  Jordan B. Pollack,et al.  Modeling Building-Block Interdependency , 1998, PPSN.

[17]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[18]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[19]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[20]  David E. Goldberg,et al.  Probabilistic Crowding: Deterministic Crowding with Probabilistic Replacement , 1999 .

[21]  Dirk Thierens,et al.  A generator for hierarchical problems , 2005, GECCO '05.

[22]  G. Harik Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms , 1997 .

[23]  Daniel E. Goldberg The design of innovation: Lessons from genetic algorithms , 1998 .

[24]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[25]  John R. Koza,et al.  Hidden Order: How Adaptation Builds Complexity. , 1995, Artificial Life.

[26]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[27]  Melanie Mitchell,et al.  What makes a problem hard for a genetic algorithm? Some anomalous results and their explanation , 1993, Machine Learning.

[28]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[29]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[30]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .