Evolutionary multilabel hyper-heuristic design

Nowadays, heuristics represent a commonly used alternative to solve complex optimization problems. This, however, has given rise to the problem of choosing the most effective heuristic for a given problem. In recent years, one of the most used strategies for this task has been the hyper-heuristics, which aim at selecting/generating heuristics to solve a wide range of optimization problems. Most of the existing selection hyper-heuristics attempt to recommend only one heuristic for a given instance. However, for some classes of problems, more than one heuristic can be suitable. With this premise, in this paper, we address this issue through an evolutionary multilabel learning approach for building hyper-heuristics. Unlike traditional approaches, in themultilabel formulation, the result could not be a single recommendation, but a set of potential heuristics. Due to the fact that cooperative coevolutionary algorithms allow us to divide the problem into several subproblems, it results in a natural approach for dealing with multilabel classification. The proposed cooperative coevolutionarymultilabel approach aims at choosing the most relevant patterns for each heuristic. For the experimental study included in this paper, we have used a set of constraint satisfaction problems as our study case. Our experimental results suggest that the proposed method is able to generate accurate hyper-heuristics that outperform reference methods.

[1]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[2]  Michèle Sebag,et al.  Online Heuristic Selection in Constraint Programming , 2009 .

[3]  Bernhard Sendhoff,et al.  Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Ujjwal Maulik,et al.  Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part II , 2014, IEEE Transactions on Evolutionary Computation.

[5]  Toby Walsh,et al.  The Constrainedness of Search , 1996, AAAI/IAAI, Vol. 1.

[6]  Hendrik Blockeel,et al.  Multiple-Instance Learning , 2010, Encyclopedia of Machine Learning.

[7]  S. García,et al.  An Extension on "Statistical Comparisons of Classifiers over Multiple Data Sets" for all Pairwise Comparisons , 2008 .

[8]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[9]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[10]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[11]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[12]  Eyke Hüllermeier,et al.  On label dependence in multilabel classification , 2010, ICML 2010.

[13]  Graham Kendall,et al.  Hyper-Heuristics: An Emerging Direction in Modern Search Technology , 2003, Handbook of Metaheuristics.

[14]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[15]  Ben Paechter,et al.  A Lifelong Learning Hyper-heuristic Method for Bin Packing , 2015, Evolutionary Computation.

[16]  James M. McCollum,et al.  A constraint satisfaction algorithm for microcontroller selection and pin assignment , 2010, Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon).

[17]  Christophe Lecoutre,et al.  Backjump-based techniques versus conflict-directed heuristics , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[18]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[19]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[20]  Hugo Terashima-Marín,et al.  Experimental Matching of Instances to Heuristics for Constraint Satisfaction Problems , 2016, Comput. Intell. Neurosci..

[21]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[22]  Everton Alvares Cherman,et al.  Incorporating label dependency into the binary relevance framework for multi-label classification , 2012, Expert Syst. Appl..

[23]  J. L. Hodges,et al.  Rank Methods for Combination of Independent Experiments in Analysis of Variance , 1962 .

[24]  Christophe Lecoutre Constraint Networks , 1992 .

[25]  Graham Kendall,et al.  A Hyperheuristic Approach to Scheduling a Sales Summit , 2000, PATAT.

[26]  Ujjwal Maulik,et al.  A Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part I , 2014, IEEE Transactions on Evolutionary Computation.

[27]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[28]  Paul Walton Purdom,et al.  Search Rearrangement Backtracking and Polynomial Average Time , 1983, Artif. Intell..

[29]  Teresa Gonçalves,et al.  A Preliminary Approach to the Multilabel Classification Problem of Portuguese Juridical Documents , 2003, EPIA.

[30]  Rina Dechter,et al.  Experimental Evaluation of Preprocessing Algorithms for Constraint Satisfaction Problems , 1994, Artif. Intell..

[31]  Kun Zhang,et al.  Multi-label learning by exploiting label dependency , 2010, KDD.

[32]  Per Kristian Lehre,et al.  Limits to Learning in Reinforcement Learning Hyper-heuristics , 2016, EvoCOP.

[33]  Rudolf Paul Wiegand,et al.  An analysis of cooperative coevolutionary algorithms , 2004 .

[34]  Lakhdar Sais,et al.  Boosting Systematic Search by Weighting Constraints , 2004, ECAI.

[35]  Witold Pedrycz,et al.  Guest Editorial: Special Issue on Advances in Multiobjective Evolutionary Algorithms for Data Mining , 2014, IEEE Trans. Evol. Comput..

[36]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[37]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[38]  Hugo Terashima-Marín,et al.  Combine and conquer: an evolutionary hyper-heuristic approach for solving constraint satisfaction problems , 2016, Artificial Intelligence Review.

[39]  A. B. Petrochenkov,et al.  Structural synthesis of complex electrotechnical equipment on the basis of the constraint satisfaction method , 2015 .