A population-based clustering technique using particle swarm optimization and k-means

A population-based clustering technique, which attempts to integrate different particle swarm optimizers (PSOs) with the famous k-means algorithm, is proposed. More specifically, six existing extensively studied PSOs, which have shown promising performance for continuous optimization, are hybridized separately with Lloyd’s k-means algorithm, leading to six PSO-based clustering methods. These PSO-based approaches use different social communications among neighbors to make some particles escape from local optima to enhance exploration, while k-means is utilized to refine the partitioning results for accelerating convergence. Comparative experiments on 12 synthetic and real-life datasets show that the proposed population-based clustering technique can obtain better and more stable solutions than five individual-based counterparts in most cases. Further, the effects of four different population topologies, three kinds of parameter settings, and two types of initialization methods on the clustering performance are empirically investigated. Moreover, seven boundary handling strategies for PSOs are firstly summarized. Finally, some unexpected conclusions are drawn from the experiments.

[1]  Erwie Zahara,et al.  A hybridized approach to data clustering , 2008, Expert Syst. Appl..

[2]  Patricio A. Vela,et al.  A Comparative Study of Efficient Initialization Methods for the K-Means Clustering Algorithm , 2012, Expert Syst. Appl..

[3]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[4]  Andries Petrus Engelbrecht,et al.  Dynamic clustering using particle swarm optimization with application in image segmentation , 2006, Pattern Analysis and Applications.

[5]  Fakhri Karray,et al.  Flocking based approach for data clustering , 2010, Natural Computing.

[6]  Jing J. Liang,et al.  Hybrid Bacterial Foraging Algorithm for Data Clustering , 2013, IDEAL.

[7]  Michael K. Ng,et al.  Automated variable weighting in k-means type clustering , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Xiao-Feng Xie,et al.  Handling boundary constraints for numerical optimization by particle swarm flying in periodic search space , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[9]  Dervis Karaboga,et al.  A novel clustering approach: Artificial Bee Colony (ABC) algorithm , 2011, Appl. Soft Comput..

[10]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[11]  Sivakumar Ramakrishnan,et al.  A survey: hybrid evolutionary algorithms for cluster analysis , 2011, Artificial Intelligence Review.

[12]  Andries Petrus Engelbrecht,et al.  Data clustering using particle swarm optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[13]  Ying Lin,et al.  Particle Swarm Optimization With an Aging Leader and Challengers , 2013, IEEE Transactions on Evolutionary Computation.

[14]  Y. Rahmat-Samii,et al.  Particle swarm optimization in electromagnetics , 2004, IEEE Transactions on Antennas and Propagation.

[15]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[16]  G. W. Milligan,et al.  An examination of procedures for determining the number of clusters in a data set , 1985 .

[17]  Stan Matwin,et al.  A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data , 2013, Artificial Intelligence Review.

[18]  Erkki Oja,et al.  Improving cluster analysis by co-initializations , 2014, Pattern Recognit. Lett..

[19]  Ujjwal Maulik,et al.  Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part II , 2014, IEEE Transactions on Evolutionary Computation.

[20]  C. A. Murthy,et al.  In search of optimal clusters using genetic algorithms , 1996, Pattern Recognit. Lett..

[21]  Chu-Sing Yang,et al.  A fast particle swarm optimization for clustering , 2015, Soft Comput..

[22]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[23]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[24]  Chi-Yang Tsai,et al.  Particle swarm optimization with selective particle regeneration for data clustering , 2011, Expert Syst. Appl..

[25]  Wenquan Chen,et al.  Cluster analysis based on attractor particle swarm optimization with boundary zoomed for working conditions classification of power plant pulverizing system , 2013, Neurocomputing.

[26]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[27]  Li-Yeh Chuang,et al.  Chaotic particle swarm optimization for data clustering , 2011, Expert Syst. Appl..

[28]  Leandro Nunes de Castro,et al.  The proposal of a velocity memoryless clustering swarm , 2010, IEEE Congress on Evolutionary Computation.

[29]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[30]  Pascal Bouvry,et al.  Particle swarm optimization: Hybridization perspectives and experimental illustrations , 2011, Appl. Math. Comput..

[31]  Ching-Yi Chen,et al.  Particle swarm optimization algorithm and its application to clustering analysis , 2004, 2012 Proceedings of 17th Conference on Electrical Power Distribution.

[32]  Erik K. Antonsson,et al.  Dynamic partitional clustering using evolution strategies , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[33]  Alex Alves Freitas,et al.  A Survey of Evolutionary Algorithms for Clustering , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[34]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[35]  D. Pham,et al.  Selection of K in K-means clustering , 2005 .

[36]  Aristidis Likas,et al.  The MinMax k-Means clustering algorithm , 2014, Pattern Recognit..

[37]  Xiao-Feng Xie,et al.  DEPSO: hybrid particle swarm with differential evolution operator , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[38]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[39]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[40]  Marco Dorigo,et al.  Ant-Based Clustering and Topographic Mapping , 2006, Artificial Life.

[41]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[43]  Gillian Dobbie,et al.  Particle Swarm Optimization Based Hierarchical Agglomerative Clustering , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[44]  Michael J. Laszlo,et al.  A genetic algorithm that exchanges neighboring centers for k-means clustering , 2007, Pattern Recognit. Lett..

[45]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[46]  Leandro N. de Castro,et al.  Data Clustering with Particle Swarms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[47]  Saeed Jalili,et al.  Dynamic clustering using combinatorial particle swarm optimization , 2012, Applied Intelligence.

[48]  Pedro Larrañaga,et al.  An empirical comparison of four initialization methods for the K-Means algorithm , 1999, Pattern Recognit. Lett..

[49]  Taher Niknam,et al.  An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis , 2010, Appl. Soft Comput..

[50]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[51]  Paul S. Bradley,et al.  Refining Initial Points for K-Means Clustering , 1998, ICML.

[52]  Lawrence W. Lan,et al.  Genetic clustering algorithms , 2001, Eur. J. Oper. Res..

[53]  Swagatam Das,et al.  Automatic Clustering Using an Improved Differential Evolution Algorithm , 2007 .

[54]  Andries P. Engelbrecht,et al.  Image Classification using Particle Swarm Optimization , 2002, SEAL.

[55]  André Hardy,et al.  An examination of procedures for determining the number of clusters in a data set , 1994 .

[56]  Gillian Dobbie,et al.  An Evolutionary Particle Swarm Optimization algorithm for data clustering , 2008, 2008 IEEE Swarm Intelligence Symposium.

[57]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[58]  T. Huang,et al.  A hybrid boundary condition for robust particle swarm optimization , 2005, IEEE Antennas and Wireless Propagation Letters.

[59]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[60]  Wei Chu,et al.  Handling boundary constraints for particle swarm optimization in high-dimensional search space , 2011, Inf. Sci..

[61]  Hung T. Nguyen,et al.  Data Clustering Using Variants of Rapid Centroid Estimation , 2014, IEEE Transactions on Evolutionary Computation.

[62]  Ujjwal Maulik,et al.  A Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part I , 2014, IEEE Transactions on Evolutionary Computation.