Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms

Learning Gene Linkage to E ciently Solve Problems of Bounded Di culty Using Genetic Algorithms by Georges Raif Harik Co-Chairs: Keki B. Irani and David E. Goldberg The complicated nature of modern scienti c endeavors often times requires the employment of black-box optimization. For the past twenty years, the simple genetic algorithm (sGA) has proven to be a fertile inspiration for such techniques. Yet, many attempts to improve or adapt the sGA remain disconnected with its prevailing theory. This theory suggests that the sGA works by propagating building blocks|highly t similarities in the structure of its solutions|and that it can fail by not recombining these building blocks in one optimal solution. The most successful of previous attempts to facilitate building block recombination have strayed far from the operation of the sGA, resulting in techniques that are di cult to use and implement. This dissertation presents an approach to solving the recombination problem without straying too far from the spirit of the sGA. By learning linkage, which brings the genes that constitute a building block closer together, this approach retains the sGA's operations of crossover and selection. However, this linkage-learning genetic algorithm (LLGA) can only be successful by controlling the forces of selection. It does this by shedding the deterministic mapping present in the sGA between chromosomes and solutions. The resulting algorithm is shown through both theoretical time complexity analysis and experimental veri cation to e ciently solve a large class of problems that are di cult for the sGA. c Georges Raif Harik 1997 All Rights Reserved This dissertation is dedicated to those whose wisdom I hope to carry with me forever. Nazira Issa, Bahiya Harik, Naoum Issa, Khairallah Harik, Jamil Harik, Pierre Issa and Hanna El-Murr. ii ACKNOWLEDGEMENTS I would like to take this opportunity to thank those who have helped me in my quest to learn. My parents, Raif and Hanan Harik, instilled in me a desire to know, and undertook great sacri ces to allow me to do so. For these, I can never repay them, but would like to thank them. My brothers, Pierre and Ralph, and my sister Nadine, provide the joy that is the fuel for my desire and perseverance. Thank you. Professor David Goldberg has taught me more than anyone what diligent work means. I have learned much from working with him over these last ve years, and hope to continue to do so in the future. Professor Keki Irani has supported me in my e orts dating back to my undergraduate days. I would like to thank them, and Professors Robert Lindsay, and Edmund Durfee, for overseeing my dissertation. In my time at the Illinois Genetic Algorithms Lab, I have met many outstanding people whom I now consider to be friends. I would like to thank (in no particular order) Je Horn, Hillol Kargupta, Ducky Sherwood, Brad Miller, Erick Cantu-Paz, Liwei Wang, Fernando Lobo, Samir Mahfoud, Angela Pereira, Dirk Thierens and Jason Wilcox for their company. Many people have gone beyond the call of duty in helping me prepare this nal manuscript. I would like to thank (again in no particular order) Staci Young, Kevin Carmody, Cynthia Han, Jonathan March, Donna Eiskamp, Sheryl Hembrey, Kerry Zakrzewski, Mimi Valentic, Mary Bedy and my uncle Gabriel Issa for their help. Finally, I would like to thank my uncle Pierre Issa and the Detroit Children's hospital for helping give me the chance to be here. This study was sponsored by the Air Force O ce of Scienti c Research, Air Force Materiel Command, USAF, under grants number F49620-94-1-0103, F49620-95-1-0338, and F49620-97-1-0050. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the author and should not be interpreted as necessarily representing the o cial policies or endorsements, either expressed or implied, of the Air Force O ce of Scienti c Research or the U.S. Government. iii TABLE OF CONTENTS DEDICATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii LIST OF APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi CHAPTERS

[1]  R. A. Fisher,et al.  The Genetical Theory of Natural Selection , 1931 .

[2]  H. Geiringer On the Probability Theory of Linkage in Mendelian Heredity , 1944 .

[3]  John Daniel. Bagley,et al.  The behavior of adaptive systems which employ genetic and correlation algorithms : technical report , 1967 .

[4]  R. Rosenberg Simulation of genetic populations with biochemical properties : technical report , 1967 .

[5]  D. J. Cavicchio,et al.  Adaptive search using simulated evolution , 1970 .

[6]  Daniel Raymond Frantz,et al.  Nonlinearities in genetic adaptive search. , 1972 .

[7]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Albert Donally Bethke,et al.  Genetic Algorithms as Function Optimizers , 1980 .

[10]  David E. Goldberg,et al.  Alleles, loci and the traveling salesman problem , 1985 .

[11]  Lawrence Davis,et al.  Applying Adaptive Algorithms to Epistatic Domains , 1985, IJCAI.

[12]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[13]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[14]  J. David Schaffer,et al.  An Adaptive Crossover Distribution Mechanism for Genetic Algorithms , 1987, ICGA.

[15]  D. Ackley A connectionist machine for genetic hillclimbing , 1987 .

[16]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[17]  Kalyanmoy Deb,et al.  An Investigation of Niche and Species Formation in Genetic Function Optimization , 1989, ICGA.

[18]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[19]  Kenneth A. De Jong,et al.  An Analysis of Multi-Point Crossover , 1990, FOGA.

[20]  Kenneth A. De Jong,et al.  An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms , 1990, PPSN.

[21]  Larry J. Eshelman,et al.  The CHC Adaptive Search Algorithm: How to Have Safe Search When Engaging in Nontraditional Genetic Recombination , 1990, FOGA.

[22]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[23]  James R. Levenick Inserting Introns Improves Genetic Algorithm Success Rate: Taking a Cue from Biology , 1991, ICGA.

[24]  Samir W. Mahfoud Crowding and Preselection Revisited , 1992, PPSN.

[25]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[26]  Kalyanmoy Deb,et al.  Analyzing Deception in Trap Functions , 1992, FOGA.

[27]  Lashon B. Booker,et al.  Recombination Distributions for Genetic Algorithms , 1992, FOGA.

[28]  Dirk Thierens,et al.  Mixing in Genetic Algorithms , 1993, ICGA.

[29]  Dirk Thierens,et al.  Convergence Models of Genetic Algorithm Selection Schemes , 1994, PPSN.

[30]  G. Harik Finding Multiple Solutions in Problems of Bounded Diiculty Finding Multiple Solutions in Problems of Bounded Diiculty , 1994 .

[31]  W. Spears,et al.  On the Virtues of Parameterized Uniform Crossover , 1995 .

[32]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[33]  Jan Paredis,et al.  The Symbiotic Evolution of Solutions and Their Representations , 1995, International Conference on Genetic Algorithms.

[34]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[35]  Jim Smith,et al.  Recombination strategy adaptation via evolution of gene linkage , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[36]  H. Kargupta Search, polynomial complexity, and the fast messy genetic algorithm , 1996 .

[37]  Samir W. Mahfoud Niching methods for genetic algorithms , 1996 .

[38]  David E. Goldberg,et al.  The gambler''s ruin problem , 1997 .

[39]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.