A Population-Based Clustering Technique Using Particle Swarm Optimization and K-Means

A population-based clustering technique, which attempts to integrate different particle swarm optimizers (PSOs) with the famous k-means algorithm, is proposed. More specifically, six existing extensively studied PSOs, which have shown promising performance for continuous optimization, are hybridized separately with Lloyd's k-means algorithm, leading to six PSO-based clustering methods. These PSO-based approaches use different social communications among neighbors to make some particles escape from local optima to enhance exploration, while k-means is utilized to refine the partitioning results for accelerating convergence. Comparative experiments on 12 synthetic and real-life datasets show that the proposed population-based clustering technique can obtain better and more stable solutions than five individual-based counterparts in most cases. Further, the effects of four different population topologies, three kinds of parameter settings, and two types of initialization methods on the clustering performance are empirically investigated. Moreover, seven boundary handling strategies for PSOs are firstly summarized. Finally, some unexpected conclusions are drawn from the experiments.

[1]  Michael K. Ng,et al.  Automated variable weighting in k-means type clustering , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Xiao-Feng Xie,et al.  Handling boundary constraints for numerical optimization by particle swarm flying in periodic search space , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[3]  Dervis Karaboga,et al.  A novel clustering approach: Artificial Bee Colony (ABC) algorithm , 2011, Appl. Soft Comput..

[4]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[5]  C. A. Murthy,et al.  In search of optimal clusters using genetic algorithms , 1996, Pattern Recognit. Lett..

[6]  Anil K. Jain Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[7]  Patricio A. Vela,et al.  A Comparative Study of Efficient Initialization Methods for the K-Means Clustering Algorithm , 2012, Expert Syst. Appl..

[8]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[9]  Ching-Yi Chen,et al.  Particle swarm optimization algorithm and its application to clustering analysis , 2004, 2012 Proceedings of 17th Conference on Electrical Power Distribution.

[10]  Alex Alves Freitas,et al.  A Survey of Evolutionary Algorithms for Clustering , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[11]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[12]  Stan Matwin,et al.  A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data , 2013, Artificial Intelligence Review.

[13]  Lawrence W. Lan,et al.  Genetic clustering algorithms , 2001, Eur. J. Oper. Res..

[14]  Swagatam Das,et al.  Automatic Clustering Using an Improved Differential Evolution Algorithm , 2007 .

[15]  Erwie Zahara,et al.  A hybridized approach to data clustering , 2008, Expert Syst. Appl..

[16]  Andries P. Engelbrecht,et al.  Image Classification using Particle Swarm Optimization , 2002, SEAL.

[17]  Jing J. Liang,et al.  Hybrid Bacterial Foraging Algorithm for Data Clustering , 2013, IDEAL.

[18]  Rajesh Kumar,et al.  A review on particle swarm optimization algorithms and their applications to data clustering , 2011, Artificial Intelligence Review.

[19]  Y. Rahmat-Samii,et al.  Particle swarm optimization in electromagnetics , 2004, IEEE Transactions on Antennas and Propagation.

[20]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[21]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[22]  Leandro N. de Castro,et al.  Data Clustering with Particle Swarms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[23]  Saeed Jalili,et al.  Dynamic clustering using combinatorial particle swarm optimization , 2012, Applied Intelligence.

[24]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[25]  Pedro Larrañaga,et al.  An empirical comparison of four initialization methods for the K-Means algorithm , 1999, Pattern Recognit. Lett..

[26]  Andries Petrus Engelbrecht,et al.  Dynamic clustering using particle swarm optimization with application in image segmentation , 2006, Pattern Analysis and Applications.

[27]  Chi-Yang Tsai,et al.  Particle swarm optimization with selective particle regeneration for data clustering , 2011, Expert Syst. Appl..

[28]  Taher Niknam,et al.  An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis , 2010, Appl. Soft Comput..

[29]  G. W. Milligan,et al.  An examination of procedures for determining the number of clusters in a data set , 1985 .

[30]  T. Huang,et al.  A hybrid boundary condition for robust particle swarm optimization , 2005, IEEE Antennas and Wireless Propagation Letters.

[31]  Xiao-Feng Xie,et al.  DEPSO: hybrid particle swarm with differential evolution operator , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[32]  Marco Dorigo,et al.  Ant-Based Clustering and Topographic Mapping , 2006, Artificial Life.

[33]  Gillian Dobbie,et al.  An Evolutionary Particle Swarm Optimization algorithm for data clustering , 2008, 2008 IEEE Swarm Intelligence Symposium.

[34]  Jacek M. Zurada,et al.  Swarm and Evolutionary Computation , 2012, Lecture Notes in Computer Science.

[35]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[36]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[37]  Wenquan Chen,et al.  Cluster analysis based on attractor particle swarm optimization with boundary zoomed for working conditions classification of power plant pulverizing system , 2013, Neurocomputing.

[38]  Aristidis Likas,et al.  The MinMax k-Means clustering algorithm , 2014, Pattern Recognit..

[39]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[40]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[41]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[42]  Gillian Dobbie,et al.  Research on particle swarm optimization based clustering: A systematic review of literature and techniques , 2014, Swarm Evol. Comput..

[43]  Sivakumar Ramakrishnan,et al.  A survey: hybrid evolutionary algorithms for cluster analysis , 2011, Artificial Intelligence Review.

[44]  Andries Petrus Engelbrecht,et al.  Data clustering using particle swarm optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[45]  Pascal Bouvry,et al.  Particle swarm optimization: Hybridization perspectives and experimental illustrations , 2011, Appl. Math. Comput..

[46]  Fakhri Karray,et al.  Flocking based approach for data clustering , 2010, Natural Computing.

[47]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[48]  Paul S. Bradley,et al.  Refining Initial Points for K-Means Clustering , 1998, ICML.

[49]  Huiying Zhang,et al.  Research on Application of Clustering Algorithm Based on PSO for the Web Usage Pattern , 2007, 2007 International Conference on Wireless Communications, Networking and Mobile Computing.

[50]  Ying Lin,et al.  Particle Swarm Optimization With an Aging Leader and Challengers , 2013, IEEE Transactions on Evolutionary Computation.

[51]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[52]  Wei Chu,et al.  Handling boundary constraints for particle swarm optimization in high-dimensional search space , 2011, Inf. Sci..

[53]  Hung T. Nguyen,et al.  Data Clustering Using Variants of Rapid Centroid Estimation , 2014, IEEE Transactions on Evolutionary Computation.

[54]  Ujjwal Maulik,et al.  A Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part I , 2014, IEEE Transactions on Evolutionary Computation.

[55]  Di Yuan,et al.  K-means Optimization Clustering Algorithm Based on Particle Swarm Optimization and Multiclass Merging , 2012 .

[56]  Li-Yeh Chuang,et al.  Chaotic particle swarm optimization for data clustering , 2011, Expert Syst. Appl..

[57]  Leandro Nunes de Castro,et al.  The proposal of a velocity memoryless clustering swarm , 2010, IEEE Congress on Evolutionary Computation.

[58]  Erik K. Antonsson,et al.  Dynamic partitional clustering using evolution strategies , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[59]  D. Pham,et al.  Selection of K in K-means clustering , 2005 .

[60]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[61]  Gillian Dobbie,et al.  Particle Swarm Optimization Based Hierarchical Agglomerative Clustering , 2010, 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.

[62]  Michael J. Laszlo,et al.  A genetic algorithm that exchanges neighboring centers for k-means clustering , 2007, Pattern Recognit. Lett..

[63]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Erkki Oja,et al.  Improving cluster analysis by co-initializations , 2014, Pattern Recognit. Lett..

[65]  Chu-Sing Yang,et al.  A fast particle swarm optimization for clustering , 2015, Soft Comput..