Flex-GP: Genetic Programming on the Cloud

We describe Flex-GP, which we believe to be the first largescale genetic programming cloud computing system. We took advantage of existing software and selected a socket-based, client-server architecture and an island-based distribution model. We developed core components required for deployment on Amazon's EC2. Scaling the system to hundreds of nodes presented several unexpected challenges and required the development of software for automatically managing deployment, reporting, and error handling. The system's performance was evaluated on two metrics, performance and speed, on a difficult symbolic regression problem. Our largest successful Flex-GP runs reached 350 nodes and taught us valuable lessons for the next phase of scaling.

[1]  Marco Tomassini,et al.  Spatially Structured Evolutionary Algorithms: Artificial Evolution in Space and Time (Natural Computing Series) , 2005 .

[2]  Ben Paechter,et al.  Empirical Validation of a Gossiping Communication Mechanism for Parallel EAs , 2007, EvoWorkshops.

[3]  Dick den Hertog,et al.  Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via Pareto Genetic Programming , 2009, IEEE Transactions on Evolutionary Computation.

[4]  Prashant Pandey,et al.  Cloud computing , 2010, ICWET.

[5]  Xavier Llorà,et al.  Scaling Genetic Algorithms Using MapReduce , 2009, 2009 Ninth International Conference on Intelligent Systems Design and Applications.

[6]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .

[7]  Leonardo Vanneschi,et al.  Theory and practice for efficient genetic programming , 2004 .

[8]  Grant Dick,et al.  Spatially-Structured Evolutionary Algorithms and Sharing: Do They Mix? , 2006, SEAL.

[9]  Erick Cantú-Paz,et al.  A Survey of Parallel Genetic Algorithms , 2000 .

[10]  Una-May O'Reilly,et al.  Evolutionary and Generative Music Informs Music HCI - And Vice Versa , 2013, Music and Human-Computer Interaction.

[11]  Brian Hayes,et al.  What Is Cloud Computing? , 2019, Cloud Technologies.

[12]  Rolf Drechsler,et al.  Applications of Evolutionary Computing, EvoWorkshops 2008: EvoCOMNET, EvoFIN, EvoHOT, EvoIASP, EvoMUSART, EvoNUM, EvoSTOC, and EvoTransLog, Naples, Italy, March 26-28, 2008. Proceedings , 2008, EvoWorkshops.

[13]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[14]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .