On the Unification of k-Harmonic Means and Fuzzy c-Means Clustering Problems under Kernelization

This paper presents a common algorithm for the kernel k-harmonic means (KKHM) and the kernel fuzzy c-means (KFCM) clustering problems. We incorporate kernel functions in a generalized fuzzy c- means cost function, forming the cost function of a kernelized general fuzzy c-means (KGFCM) problem, and design an algorithm to locally minimize this cost function. The KGFCM cost function has two parameters: the exponent p of the Euclidean distance, and the fuzzy weighting exponent m. By setting proper values for p and m in our algorithm, one can execute the KKHM or the KFCM algorithm. Using the algorithm for KKHM, we compare its clustering performance with the popular kernel k-means and KFCM algorithms. Experiments performed on real-world and synthetic datasets show the superior clustering capabilities of KKHM. We also show that KKHM retains the advantages of the original KHM algorithm, resulting in better clustering performance when high number of clusters are present.

[1]  Jens Jäkel,et al.  A New Convergence Proof of Fuzzy c-Means , 2005, IEEE Trans. Fuzzy Syst..

[2]  Guillaume Cleuziou,et al.  Kernel methods for point symmetry-based clustering , 2015, Pattern Recognit..

[3]  Francisco de A. T. de Carvalho,et al.  Kernel-based hard clustering methods with kernelization of the metric and automatic weighting of the variables , 2016, Pattern Recognit..

[4]  Nikhil R. Pal,et al.  What and When Can We Gain From the Kernel Versions of C-Means Algorithm? , 2014, IEEE Transactions on Fuzzy Systems.

[5]  Inderjit S. Dhillon,et al.  Kernel k-means: spectral clustering and normalized cuts , 2004, KDD.

[6]  Jian Yu,et al.  Optimality test for generalized FCM and its application to parameter selection , 2005, IEEE Transactions on Fuzzy Systems.

[7]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[8]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[9]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[10]  Dao-Qiang Zhang,et al.  A novel kernelized fuzzy C-means algorithm with application in medical image segmentation , 2004, Artif. Intell. Medicine.

[11]  Bin Zhang,et al.  Genera lized K- Harmonic Means - - Boosting in Unsupervised Learnin g , 2000 .

[12]  Miin-Shen Yang,et al.  A Gaussian kernel-based fuzzy c-means algorithm with a spatial bias correction , 2008, Pattern Recognit. Lett..

[13]  Witold Pedrycz,et al.  Kernel-based fuzzy clustering and fuzzy clustering: A comparative experimental study , 2010, Fuzzy Sets Syst..

[14]  L. Hubert,et al.  Comparing partitions , 1985 .

[15]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.

[16]  Doheon Lee,et al.  A kernel-based subtractive clustering method , 2005, Pattern Recognit. Lett..

[17]  Colin Fyfe,et al.  The kernel self-organising map , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[18]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[19]  Greg Hamerly,et al.  Alternatives to the k-means algorithm that find better clusterings , 2002, CIKM '02.

[20]  Rong Zhang,et al.  A large scale clustering scheme for kernel K-Means , 2002, Object recognition supported by user interaction for service robots.

[21]  Bin Zhang,et al.  Generalized K-Harmonic Means - Dynamic Weighting of Data in Unsupervised Learning , 2001, SDM.

[22]  Bin Zhang Generalized K-Harmonic Means -- Boosting in Unsupervised Learning , 2000 .

[23]  Douglas Steinley,et al.  K-means clustering: a half-century synthesis. , 2006, The British journal of mathematical and statistical psychology.

[24]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[25]  N. Mitianoudis,et al.  Spatial kernel K-harmonic means clustering for multi-spectral image segmentation , 2007 .

[26]  Umeshwar Dayal,et al.  K-Harmonic Means - A Data Clustering Algorithm , 1999 .

[27]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[28]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[29]  Francisco de A. T. de Carvalho,et al.  Kernel fuzzy c-means with automatic variable weighting , 2014, Fuzzy Sets Syst..

[30]  Nikhil R. Pal,et al.  Is it rational to partition a data set using kernel-clustering? , 2011, 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011).

[31]  Xiao-bin Zhi,et al.  Some Notes on K-Harmonic Means Clustering Algorithm , 2010 .