Differential evolution and particle swarm optimisation in partitional clustering

Many partitional clustering algorithms based on genetic algorithms (GA) have been proposed to tackle the problem of finding the optimal partition of a data set. Very few studies considered alternative stochastic search heuristics other than GAs or simulated annealing. Two promising algorithms for numerical optimisation, which are hardly known outside the search heuristics field, are particle swarm optimisation (PSO) and differential evolution (DE). The performance of GAs for a representative point evolution approach to clustering is compared with PSO and DE. The empirical results show that DE is clearly and consistently superior compared to GAs and PSO for hard clustering problems, both with respect to precision as well as robustness (reproducibility) of the results. Only for simple data sets, the GA and PSO can obtain the same quality of results. Apart from superior performance, DE is easy to implement and requires hardly any parameter tuning compared to substantial tuning for GAs and PSOs. Our study shows that DE rather than GAs should receive primary attention in partitional clustering algorithms.

[1]  T. Krink,et al.  Extending particle swarm optimisers with self-organized criticality , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  Dr. Zbigniew Michalewicz,et al.  How to Solve It: Modern Heuristics , 2004 .

[3]  David B. Fogel,et al.  Evolutionary Computation: The Fossil Record , 1998 .

[4]  F. Marriott Practical problems in a method of cluster analysis. , 1971, Biometrics.

[5]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[6]  Sanghamitra Bandyopadhyay,et al.  Pattern classification with genetic algorithms , 1995, Pattern Recognit. Lett..

[7]  Ujjwal Maulik,et al.  An evolutionary technique based on K-Means algorithm for optimal clustering in RN , 2002, Inf. Sci..

[8]  Vijay V. Raghavan,et al.  A clustering strategy based on a formalism of the reproductive process in natural systems , 1979, SIGIR '79.

[9]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[10]  Ujjwal Maulik,et al.  Fuzzy partitioning using a real-coded variable-length genetic algorithm for pixel classification , 2003, IEEE Trans. Geosci. Remote. Sens..

[11]  Ludmila I. Kuncheva,et al.  Editing for the k-nearest neighbors rule by a genetic algorithm , 1995, Pattern Recognit. Lett..

[12]  Gary B. Fogel,et al.  Noisy optimization problems - a particular challenge for differential evolution? , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[13]  R. Thomsen Flexible ligand docking using differential evolution , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[14]  C. A. Murthy,et al.  In search of optimal clusters using genetic algorithms , 1996, Pattern Recognit. Lett..

[15]  Ujjwal Maulik,et al.  Genetic algorithm-based clustering technique , 2000, Pattern Recognit..

[16]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[17]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[18]  Lin-Yu Tseng,et al.  A genetic approach to the automatic clustering problem , 2001, Pattern Recognit..

[19]  Gérard Govaert,et al.  Gaussian parsimonious clustering models , 1995, Pattern Recognit..

[20]  Sandra Paterlini,et al.  Evolutionary Approaches for Cluster Analysis , 2003 .

[21]  Emanuel Falkenauer,et al.  Genetic Algorithms and Grouping Problems , 1998 .

[22]  T. Krink,et al.  Particle swarm optimisation with spatial particle extension , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[23]  Sandra Paterlini,et al.  Evolutionary approaches for statistical modelling , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[24]  René Thomsen,et al.  A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[25]  Kenneth V. Price,et al.  An introduction to differential evolution , 1999 .

[26]  Sanghamitra Bandyopadhyay,et al.  Pattern classification using genetic algorithms: Determination of H , 1998, Pattern Recognit. Lett..

[27]  P. Sopp Cluster analysis. , 1996, Veterinary immunology and immunopathology.

[28]  R. Storn,et al.  Differential evolution a simple and efficient adaptive scheme for global optimization over continu , 1997 .

[29]  Vijay V. Raghavan,et al.  A clustering strategy based on a formalism of the reproductive process in natural systems , 1979, SIGIR 1979.

[30]  Russell C. Eberhart,et al.  Gene clustering using self-organizing maps and particle swarm optimization , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[31]  P. Vadstrup,et al.  Parameter identification of induction motors using differential evolution , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[32]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[33]  A. Raftery,et al.  Model-based Gaussian and non-Gaussian clustering , 1993 .

[34]  M. Narasimha Murty,et al.  Genetic K-means algorithm , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[35]  Ali M. S. Zalzala,et al.  A genetic rule-based data clustering toolkit , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[36]  E. Forgy Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[37]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[38]  F. Marriott Optimization methods of cluster analysis , 1982 .

[39]  Riccardo Poli,et al.  Particle Swarm Optimisation , 2011 .

[40]  Sanghamitra Bandyopadhyay,et al.  Theoretical performance of genetic pattern classifier , 1999 .

[41]  Roy George,et al.  A variable-length genetic algorithm for clustering and classification , 1995, Pattern Recognit. Lett..

[42]  Francesco Masulli,et al.  Soft Computing Applications , 2003 .

[43]  Sanghamitra Bandyopadhyay,et al.  Simulated Annealing Based Pattern Classification , 1998, Inf. Sci..

[44]  Lawrence W. Lan,et al.  Genetic clustering algorithms , 2001, Eur. J. Oper. Res..

[45]  Ujjwal Maulik,et al.  Genetic clustering for automatic evolution of clusters and application to image classification , 2002, Pattern Recognit..

[46]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .