Search, polynomial complexity, and the fast messy genetic algorithm

Blackbox optimization--optimization in presence of limited knowledge about the objective function--has recently enjoyed a large increase in interest because of the demand from the practitioners. This has triggered a race for new high performance algorithms for solving large, difficult problems. Simulated annealing, genetic algorithms, tabu search are some examples. Unfortuntely, each of these algorithms is creating a separate field in itself and their use in practice is often guided by personal discretion rather than scientific reasons. The primary reason behind this confusing situation is the lack of any comprehensive understanding about blackbox search. This dissertation takes a step toward clearing some of the confusion. The main objectives of this dissertation are: (1) present SEARCH (Search Envisioned As Relation & Class Hierarchizing)--an alternate perspective of blackbox optimization and its quantitative analysis that lays the foundation essential for transcending the limits of random enumerative search; (2) design and testing of the fast messy genetic algorithm. SEARCH is a general framework for understanding blackbox optimization in terms of relations, classes and ordering. The primary motivation comes from the observation that sampling in blackbox optimization is essentially an inductive process (Michalski, 1983) and in absence of any relation among the members of the search space, induction is no better than enumeration. The foundation of SEARCH is laid on a decomposition of BBO into relation, class, and sample spaces. An ordinal, probablistic, and approximate framework is developed on this foundation to identify the fundamental principles in-blackbox optimization, essential for transcending the limits of random enumerative search. Bounds on success probability and sample complexity ate derived. I explicitly consider specific blackbox algorithms like simulated annealing, genetic algorithms and demonstrate that the fundamental computations in all of them can be captured using SEARCH. SEARCH also offers an alternate perspective of natural evolution that establishes the computational role of gene expression (DNA $\to$ RNA $\to$ Protein) in evolution. This model of evolutionary computation hypothesizes a possible mapping of the decomposition is relation, class, and sample spaces of SEARCH into the transcriptional regulatory mechanisms, proteins, and DNA respectively. The second part of this dissertation starts by noting the limitations of simple GAs, which fail to properly search for relations and makes decision making very noisy by combining relation, class, and the sample spaces. Messy genetic algorithms (Goldberg, Korb, & Deb, 1989; Deb, 1991) are a rare class of algorithms that emphasize the search for relations. Despite this strength of messy GAs, they lacked complete benefits of implicit parallelism (Holland, 1975). The fast messy GA initiated by Goldberg, Deb, Kargupta, and Harik (1993) introduced some of the benefits of implicit parallelism in messy GA without sacrificing its other strengths very much. This dissertation investigates fast messy GAs and presents test results to demonstrate its performance for order-k delineable problems.