High dimensional search-based software engineering: finding tradeoffs among 15 objectives for automating software refactoring using NSGA-III

There is a growing need for scalable search-based software engineering approaches that address software engineering problems where a large number of objectives are to be optimized. Software refactoring is one of these problems where a refactoring sequence is sought that optimizes several software metrics. Most of the existing refactoring work uses a large set of quality metrics to evaluate the software design after applying refactoring operations, but current search-based software engineering approaches are limited to using a maximum of five metrics. We propose for the first time a scalable search-based software engineering approach based on a newly proposed evolutionary optimization method NSGA-III where there are 15 different objectives to be optimized. In our approach, automated refactoring solutions are evaluated using a set of 15 distinct quality metrics. We evaluated this approach on seven large open source systems and found that, on average, more than 92% of code smells were corrected. Statistical analysis of our experiments over 31 runs shows that NSGA-III performed significantly better than two other many-objective techniques (IBEA and MOEA/D), a multi-objective algorithm (NSGA-II) and two mono-objective approaches, hence demonstrating that our NSGA-III approach represents the new state of the art in fully-automated refactoring.

[1]  Carlos A. Coello Coello,et al.  Online Objective Reduction to Deal with Many-Objective Problems , 2009, EMO.

[2]  Saúl Zapotecas Martínez,et al.  Self-adaptation Techniques Applied to Multi-Objective Evolutionary Algorithms , 2011, LION.

[3]  Gordon Fraser,et al.  Parameter tuning or default values? An empirical investigation in search-based software engineering , 2013, Empirical Software Engineering.

[4]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[5]  Houari A. Sahraoui,et al.  Design Defects Detection and Correction by Example , 2011, 2011 IEEE 19th International Conference on Program Comprehension.

[6]  Thomas J. Mowbray,et al.  AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis , 1998 .

[7]  Mark Harman,et al.  Experimental assessment of software metrics using automated refactoring , 2012, Proceedings of the 2012 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement.

[8]  Kalyanmoy Deb,et al.  Handling many-objective problems using an improved NSGA-II procedure , 2012, 2012 IEEE Congress on Evolutionary Computation.

[9]  Kalyanmoy Deb,et al.  An Improved Adaptive Approach for Elitist Nondominated Sorting Genetic Algorithm for Many-Objective Optimization , 2013, EMO.

[10]  Christian General Chair-Alba Enrique Blum,et al.  Proceedings of the 15th annual conference companion on Genetic and evolutionary computation , 2013, Annual Conference on Genetic and Evolutionary Computation.

[11]  Mark Harman,et al.  Software engineering: an ideal set of challenges for evolutionary computation , 2013, GECCO.

[12]  Günther Ruhe,et al.  Search Based Software Engineering , 2013, Lecture Notes in Computer Science.

[13]  Lothar Thiele,et al.  A Preference-Based Evolutionary Algorithm for Multi-Objective Optimization , 2009, Evolutionary Computation.

[14]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[15]  Kiyoshi Tanaka,et al.  Adaptive Control of the Number of Crossed Genes in Many-Objective Evolutionary Optimization , 2012, LION.

[16]  Khaled Ghédira,et al.  Searching for knee regions of the Pareto front using mobile reference points , 2011, Soft Comput..

[17]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[18]  Johannes Stammel,et al.  Search-based determination of refactorings for improving the class structure of object-oriented systems , 2006, GECCO.

[19]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.

[20]  Aiko Fallas Yamashita,et al.  Do code smells reflect important maintainability aspects? , 2012, 2012 28th IEEE International Conference on Software Maintenance (ICSM).

[21]  Mark Harman,et al.  Pareto optimal search based refactoring at the design level , 2007, GECCO '07.

[22]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[23]  Khaled Ghédira,et al.  Estimating nadir point in multi-objective optimization using mobile reference points , 2010, IEEE Congress on Evolutionary Computation.

[24]  Yann-Gaël Guéhéneuc,et al.  DECOR: A Method for the Specification and Detection of Code and Design Smells , 2010, IEEE Transactions on Software Engineering.

[25]  Mark Harman,et al.  Not going to take this anymore: Multi-objective overtime planning for Software Engineering projects , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[26]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[27]  Mark Kent O'Keeffe,et al.  Search-based refactoring for software maintenance , 2008, J. Syst. Softw..

[28]  Tim Menzies,et al.  On the value of user preferences in search-based software engineering: A case study in software product lines , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[29]  Soon-Thiam Khu,et al.  An Investigation on Preference Order Ranking Scheme for Multiobjective Evolutionary Optimization , 2007, IEEE Transactions on Evolutionary Computation.

[30]  Lionel C. Briand,et al.  A practical guide for using statistical tests to assess randomized algorithms in software engineering , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[31]  Eckart Zitzler,et al.  HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization , 2011, Evolutionary Computation.

[32]  Houari A. Sahraoui,et al.  Maintainability defects detection and correction: a multi-objective approach , 2013, Automated Software Engineering.

[33]  Shari Lawrence Pfleeger,et al.  Software metrics (2nd ed.): a rigorous and practical approach , 1997 .

[34]  Betty H. C. Cheng,et al.  On the use of genetic programming for automated refactoring and the introduction of design patterns , 2010, GECCO '10.

[35]  Gabriele Bavota,et al.  Detecting bad smells in source code using change history information , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[36]  Yann-Gaël Guéhéneuc,et al.  Decor: a tool for the detection of design defects , 2007, ASE.

[37]  Khaled Ghédira,et al.  The r-Dominance: A New Dominance Relation for Interactive Evolutionary Multicriteria Decision Making , 2010, IEEE Transactions on Evolutionary Computation.

[38]  Peter J. Fleming,et al.  Preference-Inspired Coevolutionary Algorithms for Many-Objective Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[39]  Tapabrata Ray,et al.  A Pareto Corner Search Evolutionary Algorithm and Dimensionality Reduction in Many-Objective Optimization Problems , 2011, IEEE Transactions on Evolutionary Computation.

[40]  Qingfu Zhang,et al.  Objective Reduction in Many-Objective Optimization: Linear and Nonlinear Algorithms , 2013, IEEE Transactions on Evolutionary Computation.

[41]  Tim Menzies,et al.  Scalable product line configuration: A straw to break the camel's back , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[42]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[43]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[44]  Yuanyuan Zhang,et al.  Search-based software engineering: Trends, techniques and applications , 2012, CSUR.

[45]  Jouni Lampinen,et al.  Ranking-Dominance and Many-Objective Optimization , 2007, 2007 IEEE Congress on Evolutionary Computation.

[46]  Hürevren Kiliç,et al.  Search-Based Parallel Refactoring Using Population-Based Direct Approaches , 2011, SSBSE.