Enhanced optimization with composite objectives and novelty pulsation

An important benefit of multi-objective search is that it maintains a diverse population of candidates, which helps in deceptive problems in particular. Not all diversity is useful, however: candidates that optimize only one objective while ignoring others are rarely helpful. A recent solution is to replace the original objectives by their linear combinations, thus focusing the search on the most useful tradeoffs between objectives. To compensate for the loss of diversity, this transformation is accompanied by a selection mechanism that favors novelty. This paper improves this approach further by introducing novelty pulsation, i.e. a systematic method to alternate between novelty selection and local optimization. In the highly deceptive problem of discovering minimal sorting networks, it finds state-of-the-art solutions significantly faster than before. In fact, our method so far has established a new world record for the 20-lines sorting network with 91comparators. In the real-world problem of stock trading, it discovers solutions that generalize significantly better on unseen data. Composite Novelty Pulsation is therefore a promising approach to solving deceptive real-world problems through multi-objective optimization.

[1]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[2]  Antoine Cully,et al.  Robots that can adapt like animals , 2014, Nature.

[3]  Jean-Baptiste Mouret,et al.  Illuminating search spaces by mapping elites , 2015, ArXiv.

[4]  Sebastian Risi,et al.  Creative Generation of 3D Objects with Deep Learning and Innovation Engines , 2016, ICCC.

[5]  Risto Miikkulainen,et al.  Enhanced Optimization with Composite Objectives and Novelty Pulsation , 2018, GPTP.

[6]  Elliot Meyerson,et al.  Discovering evolutionary stepping stones through behavior domination , 2017, GECCO.

[7]  Stéphane Doncieux,et al.  Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study , 2012, Evolutionary Computation.

[8]  Peter Schneider-Kamp,et al.  The Quest for Optimal Sorting Networks: Efficient Generation of Two-Layer Prefixes , 2014, 2014 16th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[9]  Franklin Allen,et al.  Using genetic algorithms to find technical trading rules , 1999 .

[10]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[11]  Anthony Brabazon,et al.  Biologically inspired algorithms for financial modelling , 2006, Natural computing series.

[12]  José Ignacio Hidalgo,et al.  A meta-grammatical evolutionary process for portfolio selection and trading , 2017, Genetic Programming and Evolvable Machines.

[13]  Faustino J. Gomez,et al.  When Novelty Is Not Enough , 2011, EvoApplications.

[14]  Kenneth O. Stanley,et al.  Efficiently evolving programs through the search for novelty , 2010, GECCO '10.

[15]  L. Darrell Whitley,et al.  Delta Coding: An Iterative Search Strategy for Genetic Algorithms , 1991, ICGA.

[16]  Kenneth O. Stanley,et al.  Confronting the Challenge of Quality Diversity , 2015, GECCO.

[17]  Marjan Mernik,et al.  Exploration and exploitation in evolutionary algorithms: A survey , 2013, CSUR.

[18]  Kenneth E. Batcher,et al.  Finding better sorting networks , 2009 .

[19]  A. Shamsai,et al.  Multi-objective Optimization , 2017, Encyclopedia of Machine Learning and Data Mining.

[20]  Anders Lyhne Christensen,et al.  Devising Effective Novelty Search Algorithms: A Comprehensive Empirical Study , 2015, GECCO.

[21]  Ivo Gonçalves,et al.  Balancing Learning and Overfitting in Genetic Programming with Interleaved Sampling of Training Data , 2013, EuroGP.

[22]  Joel Lehman,et al.  Overcoming deception in evolution of cognitive behaviors , 2014, GECCO.

[23]  Risto Miikkulainen,et al.  Distributed Age-Layered Novelty Search , 2016 .

[24]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[25]  Anthony Brabazon,et al.  Objective function design in a grammatical evolutionary trading system , 2010, IEEE Congress on Evolutionary Computation.

[26]  Kenneth O. Stanley,et al.  Fully Autonomous Real-Time Autoencoder-Augmented Hebbian Learning through the Collection of Novel Experiences , 2016, ALIFE.

[27]  Faustino J. Gomez,et al.  Sustaining diversity using behavioral information distance , 2009, GECCO.

[28]  Risto Miikkulainen,et al.  Using symmetry and evolutionary search to minimize sorting networks , 2013, J. Mach. Learn. Res..

[29]  Rüdiger Westermann,et al.  UberFlow: a GPU-based particle engine , 2004, SIGGRAPH '04.

[30]  Kenneth O. Stanley,et al.  Abandoning Objectives: Evolution Through the Search for Novelty Alone , 2011, Evolutionary Computation.

[31]  R. Miikkulainen,et al.  Learning Behavior Characterizations for Novelty Search , 2016, GECCO.

[32]  Kenneth O. Stanley,et al.  Exploiting Open-Endedness to Solve Problems Through the Search for Novelty , 2008, ALIFE.

[33]  Risto Miikkulainen,et al.  Enhanced Optimization with Composite Objectives and Novelty Pulsation , 2018, GPTP.

[34]  Anthony Brabazon,et al.  Adaptive Trade Execution using a Grammatical Evolution Approach , 2014 .

[35]  Kenneth O. Stanley,et al.  Beyond Open-endedness: Quantifying Impressiveness , 2012, ALIFE.

[36]  Anders Lyhne Christensen,et al.  Evolution of swarm robotics systems with novelty search , 2013, Swarm Intelligence.

[37]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[38]  Peter Schneider-Kamp,et al.  Sorting Networks: to the End and Back Again , 2015, J. Comput. Syst. Sci..

[39]  H. White,et al.  A Reality Check for Data Snooping , 2000 .

[40]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[41]  Kenneth O. Stanley,et al.  Evolving a diversity of virtual creatures through novelty search and local competition , 2011, GECCO '11.

[42]  Donald E. Knuth,et al.  The Art of Computer Programming: Volume 3: Sorting and Searching , 1998 .

[43]  Peter Krcah Combination of Novelty Search and Fitness-Based Search Applied to Robot Body-Brain Co-Evolution , 2010 .