No Free Lunch and Free Leftovers Theorems for Multiobjective Optimisation Problems

The classic NFL theorems are invariably cast in terms of single objective optimization problems. We confirm that the classic NFL theorem holds for general multiobjective fitness spaces, and show how this follows from a 'single-objective' NFL theorem. We also show that, given any particular Pareto Front, an NFL theorem holds for the set of all multiobjective problems which have that Pareto Front. It follows that, given any 'shape' or class of Pareto fronts, an NFL theorem holds for the set of all multiobjective problems in that class. These findings have salience in test function design. Such NFL results are cast in the typical context of absolute performance, assuming a performance metric which returns a value based on the result produced by a single algorithm. But, in multiobjective search we commonly use comparative metrics, which return performance measures based non-trivially on the results from two (or more) algorithms. Closely related to but extending the observations in the seminal NFL work concerning minimax distinctions between algorithms, we provide a 'Free Leftovers' theorem for comparative performance of algorithms over permutation functions; in words: over the space of permutation problems, every algorithm has some companion algorithm(s) which it outperforms, according to a certain well-behaved metric, when comparative performance is summed over all problems in the space.

[1]  L. Darrell Whitley,et al.  Functions as Permutations: Regarding No Free Lunch, Walsh Analysis and Summary Statistics , 2000, PPSN.

[2]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[3]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[4]  D. WhitleyComputer A Free Lunch Proof for Gray versus Binary Encodings , 1999 .

[5]  Steffen Christensen,et al.  What can we learn from No Free Lunch? a first attempt to characterize the concept of a searchable function , 2001 .

[6]  M. Hansen,et al.  Evaluating the quality of approximations to the non-dominated set , 1998 .

[7]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[8]  Darrell Whitley Functions as Permutations: Implications for No Free Lunch, Walsh Analysis and Statistics , 2000 .

[9]  Joshua D. Knowles Local-search and hybrid evolutionary algorithms for Pareto optimization , 2002 .

[10]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[11]  Marc Toussaint,et al.  On Classes of Functions for which No Free Lunch Results Hold , 2001, Inf. Process. Lett..

[12]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[13]  D. Corne,et al.  On Metrics for Comparing Non Dominated Sets , 2001 .

[14]  Thomas Jansen,et al.  Optimization with randomized search heuristics - the (A)NFL theorem, realistic scenarios, and difficult functions , 2002, Theor. Comput. Sci..

[15]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers , 2002 .

[16]  Patrick D. Surry,et al.  Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective , 1995, Computer Science Today.

[17]  Thomas Jansen,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods Perhaps Not a Free Lunch but at Least a Free Appetizer Perhaps Not a Free Lunch but at Least a Free Appetizer , 2022 .

[18]  L. D. Whitley,et al.  The No Free Lunch and problem description length , 2001 .

[19]  Joshua D. Knowles,et al.  On metrics for comparing nondominated sets , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[20]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .