A Discrete Particle Swarm for Multi-objective Problems in Polynomial Neural Networks used for Classification: A Data Mining Perspective

Approximating decision boundaries of large datasets to classify an unknown sample has been recognized by many researchers within the data mining community as a very promising research topic. The application of polynomial neural networks (PNNs) for the approximation of decision boundaries can be considered as a multiple criteria problem rather than as one involving a single criterion. Classification accuracy and architectural complexity can be thought of as two different conflicting objectives when using PNNs for classification tasks. Using these two metrics as the objectives for finding decision boundaries, this chapter adopts a Discrete Pareto Particle Swarm Optimization (DPPSO) method. DPPSO guides the evolution of the swarm by using the two aforementioned objectives: classification accuracy and architectural complexity. The effectiveness of this method is shown on real life datasets having non-linear class boundaries. Empirical results indicate that the performance of the proposed method is encouraging.

[1]  Aurora Trinidad Ramirez Pozo,et al.  Multiple objective particle swarm for classification-rule discovery , 2007, 2007 IEEE Congress on Evolutionary Computation.

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Jonathan E. Fieldsend,et al.  Using unconstrained elite archives for multiobjective optimization , 2003, IEEE Trans. Evol. Comput..

[4]  Edmund K. Burke,et al.  Parallel Problem Solving from Nature - PPSN IX: 9th International Conference, Reykjavik, Iceland, September 9-13, 2006, Proceedings , 2006, PPSN.

[5]  Alex Alves Freitas,et al.  A new discrete particle swarm algorithm applied to attribute selection in a bioinformatics data set , 2006, GECCO.

[6]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[7]  Kalyanmoy Deb,et al.  An Investigation of Niche and Species Formation in Genetic Function Optimization , 1989, ICGA.

[8]  Chih-Chuan Chen,et al.  A Multi-objective Particle Swarm Optimization Algorithm for Rule Discovery , 2007, Third International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP 2007).

[9]  Benjamin Van Roy,et al.  Solving Data Mining Problems Through Pattern Recognition , 1997 .

[10]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[11]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications , 2008, Natural Computing.

[12]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[13]  Jürgen Branke,et al.  About Selecting the Personal Best in Multi-Objective Particle Swarm Optimization , 2006, PPSN.

[14]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[15]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[16]  Ziyan Liu,et al.  Outlier Identification and Justification Using Multi-Objective PSO based Clustering Algorithm in Power System , 2007, 2007 5th IEEE International Conference on Industrial Informatics.

[17]  Vladimir D. Noghin.  An axiomatization of the generalized Edgeworth-Pareto principle in terms of choice functions , 2006, Math. Soc. Sci..

[18]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[19]  Hussein A. Abbass,et al.  An evolutionary artificial neural networks approach for breast cancer diagnosis , 2002, Artif. Intell. Medicine.

[20]  Hans-Peter Kriegel,et al.  Future trends in data mining , 2007, Data Mining and Knowledge Discovery.

[21]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[22]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[23]  Sung-Kwun Oh,et al.  Polynomial neural networks architecture: analysis and design , 2003, Comput. Electr. Eng..

[24]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[25]  M. A. Khanesar,et al.  A novel binary particle swarm optimization , 2007, 2007 Mediterranean Conference on Control & Automation.

[26]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[27]  Jonathan E. Fieldsend,et al.  Full Elite Sets for Multi-Objective Optimisation , 2002 .

[28]  Jonathan E. Fieldsend,et al.  A Multi-Objective Algorithm based upon Particle Swarm Optimisation, an Efficient Data Structure and , 2002 .

[29]  Aurora Trinidad Ramirez Pozo,et al.  Non-Ordered Data Mining Rules Through Multi-Objective Particle Swarm Optimization: Dealing with Numeric and Discrete Attributes , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[30]  Igor V. Tetko,et al.  Robust polynomial neural networks in quantative-structure activity relationship studies , 2003 .

[31]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[32]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[33]  C.A. Coello Coello,et al.  MOPSO: a proposal for multiple objective particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[34]  Hamparsum Bozdogan,et al.  Statistical Data Mining and Knowledge Discovery , 2004 .

[35]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[36]  Jens Gottlieb,et al.  Evolutionary Computation in Combinatorial Optimization , 2006, Lecture Notes in Computer Science.

[37]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[38]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[39]  Gregory Piatetsky-Shapiro,et al.  Data mining and knowledge discovery 1996 to 2005: overcoming the hype and moving from “university” to “business” and “analytics” , 2007, Data Mining and Knowledge Discovery.

[40]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[41]  A. Stacey,et al.  Particle swarm optimization with mutation , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[42]  Howard J. Hamilton,et al.  Knowledge discovery and measures of interest , 2001 .

[43]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[44]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[45]  Yongling Zheng,et al.  On the convergence analysis and parameter selection in particle swarm optimization , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[46]  M Reyes Sierra,et al.  Multi-Objective Particle Swarm Optimizers: A Survey of the State-of-the-Art , 2006 .

[47]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[48]  Tughrul Arslan,et al.  Proceedings Of The 2000 Congress On Evolutionary Computation , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[49]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[50]  Aurora Trinidad Ramirez Pozo,et al.  Exploring Multi-objective PSO and GRASP-PR for Rule Induction , 2008, EvoCOP.

[51]  Mohammad Ali Abido,et al.  Two-level of nondominated solutions approach to multiobjective particle swarm optimization , 2007, GECCO '07.

[52]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[53]  J. Teich,et al.  The role of /spl epsi/-dominance in multi objective particle swarm optimization methods , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[54]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[55]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[56]  A. E. Eiben,et al.  Evolutionary Programming VII , 1998, Lecture Notes in Computer Science.

[57]  Jürgen Teich,et al.  Strategies for finding good local guides in multi-objective particle swarm optimization (MOPSO) , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[58]  W. J. Welsh,et al.  Polynomial Neural Network for Linear and Non-linear Model Selection in Quantitative-Structure Activity Relationship Studies on the Internet , 2000, SAR and QSAR in environmental research.

[59]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[60]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 2000, Springer Berlin Heidelberg.

[61]  D Tomes,et al.  Physics and mathematics , 1972 .

[62]  U. M. Feyyad.  Data mining and knowledge discovery: making sense out of data , 1996 .

[63]  Leon G. Higley,et al.  Forensic Entomology: An Introduction , 2009 .

[64]  Chi-Yang Tsai,et al.  A multiple objective particle swarm optimization approach for inventory classification , 2008 .

[65]  Faouzi Ghorbel,et al.  An unsupervised and non-parametric bayesian classifier , 2003, Pattern Recognit. Lett..

[66]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[67]  Russell C. Eberhart,et al.  Multiobjective optimization using dynamic neighborhood particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[68]  Ganapati Panda,et al.  A reduced and comprehensible polynomial neural network for classification , 2008, Pattern Recognit. Lett..

[69]  Marco Laumanns,et al.  Combining Convergence and Diversity in Evolutionary Multiobjective Optimization , 2002, Evolutionary Computation.