Data mining methods for knowledge discovery in multi-objective optimization: Part B - New developments and applications

Abstract The first part of this paper served as a comprehensive survey of data mining methods that have been used to extract knowledge from solutions generated during multi-objective optimization. The current paper addresses three major shortcomings of existing methods, namely, lack of interactiveness in the objective space, inability to handle discrete variables and inability to generate explicit knowledge. Four data mining methods are developed that can discover knowledge in the decision space and visualize it in the objective space. These methods are (i) sequential pattern mining, (ii) clustering-based classification trees, (iii) hybrid learning, and (iv) flexible pattern mining. Each method uses a unique learning strategy to generate explicit knowledge in the form of patterns, decision rules and unsupervised rules. The methods are also capable of taking the decision maker’s preferences into account to generate knowledge unique to preferred regions of the objective space. Three realistic production systems involving different types of discrete variables are chosen as application studies. A multi-objective optimization problem is formulated for each system and solved using NSGA-II to generate the optimization datasets. Next, all four methods are applied to each dataset. In each application, the methods discover similar knowledge for specified regions of the objective space. Overall, the unsupervised rules generated by flexible pattern mining are found to be the most consistent, whereas the supervised rules from classification trees are the most sensitive to user-preferences.

[1]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[2]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[3]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[4]  Amos H. C. Ng,et al.  Industrial cost modelling and multi-objective optimisation for decision support in production systems development , 2013, Comput. Ind. Eng..

[5]  Amos H. C. Ng,et al.  Integration of data mining and multi-objective optimisation for decision support in production systems development , 2014, Int. J. Comput. Integr. Manuf..

[6]  Amos H. C. Ng,et al.  An Applied Framework for simulation-based multi-objective optimisation within production system development , 2011 .

[7]  Ramakrishnan Srikant,et al.  Fast algorithms for mining association rules , 1998, VLDB 1998.

[8]  Ramakrishnan Srikant,et al.  Mining Sequential Patterns: Generalizations and Performance Improvements , 1996, EDBT.

[9]  Leif Pehrsson,et al.  Manufacturing management and decision support using simulation-based multi-objective optimisation , 2013 .

[10]  Shu-Hsien Liao,et al.  Knowledge management technologies and applications - literature review from 1995 to 2002 , 2003, Expert Syst. Appl..

[11]  Averill M. Law,et al.  Simulation-based optimization , 2000, 2000 Winter Simulation Conference Proceedings (Cat. No.00CH37165).

[12]  Horst Tempelmeier,et al.  Practical considerations in the optimization of flow production systems , 2003 .

[13]  Amos H. C. Ng,et al.  Post-analysis of multi-objective optimization solutions using decision trees , 2015, Intell. Data Anal..

[14]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[15]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[16]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[17]  Ramakrishnan Srikant,et al.  Mining sequential patterns , 1995, Proceedings of the Eleventh International Conference on Data Engineering.

[18]  Kalyanmoy Deb,et al.  A review of hybrid evolutionary multiple criteria decision making methods , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[19]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[20]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[21]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[22]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[23]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[24]  Shigeru Obayashi,et al.  Multi-objective optimization and design rule mining for an aerodynamically efficient and stable centrifugal impeller with a vaned diffuser , 2010 .

[25]  Matías Urenda Moris,et al.  Facts Analyser: An Innovative Tool for Factory Conceptual Design Using Simulation , 2007 .

[26]  Kalyanmoy Deb,et al.  An integrated approach to automated innovization for discovering useful design principles: Case studies from engineering , 2014, Appl. Soft Comput..

[27]  Amos H. C. Ng,et al.  A synergy of multi-objective optimization and data mining for the analysis of a flexible flow shop , 2011 .

[28]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[29]  Matías Urenda Moris,et al.  Factory flow design and analysis using internet-enabled simulation-based optimization and automatic model generation , 2011, Proceedings of the 2011 Winter Simulation Conference (WSC).

[30]  Aravind Srinivasan,et al.  Innovization: innovating design principles through optimization , 2006, GECCO.

[31]  Kalyanmoy Deb,et al.  Knowledge Discovery in Production simulation By Interleaving Multi-Objective Optimization and Data Mining , 2012 .

[32]  Amos H. C. Ng,et al.  Practical Production Systems Optimization Using Multiple-Choice Sets and Manhattan Distance based Constraints Handling , 2014 .

[33]  Kalyanmoy Deb,et al.  Reference point based multi-objective optimization using evolutionary algorithms , 2006, GECCO.