MOEA Testing and Analysis

Regarding the scientific method of experimentation, it is desirable to construct an accurate, reliable, consistent and non-arbitrary representation of MOEA architectures and performance over sets of MOPs. In particular, through the use of standard procedures and criteria, one should attempt to minimize the influence of bias or prejudice of the experimenter when testing an MOEA hypothesis. The design of each experiment must conform then to an accepted “standard” approach as reflected in any generic scientific method. When employing this approach, the detailed design of MOEA experiments can draw heavily from outlines presented by Barr et al. (1995), and Jackson et al. (1991). These generic articles discuss computational experiment design for heuristic methods, providing guidelines for reporting results and ensuring their reproducibility. Specifically, they suggest that a well-designed experiment follow these steps: (1) Define experimental goals; (2) Choose measures of performance (metrics); (3) Design and execute the experiment; (4) Analyze data and draw conclusions; and (5) Report experimental results. This chapter applies these concepts in developing experimental MOEA testing procedures using appropriate MOP test suites from Chapter 3.