Geometric divergence based fuzzy clustering with strong resilience to noise features

Development of FCM with geometric divergence measure.Investigating sub-optimization problems constituting the Alternating Optimization.Performance comparison with FCM and weighted FCM.Detailed theoretical proof of strong resilience to noise features.Experimental proof of strong resilience to noise features. In this article we consider the problem of fuzzy partitional clustering using a separable multi-dimensional version of the geometric distance which includes f-divergences as special cases. We propose an iterative relocation algorithm for the Fuzzy C Means (FCM) clustering that is guaranteed to converge to local minima. We also demonstrate, through theoretical analysis, that the FCM clustering with the proposed divergence based similarity measure, is more robust towards the perturbation of noise features than the standard FCM with Euclidean distance based similarity measure. In addition, we show that FCM with the suggested geometric divergence measure has better or comparable clustering performance to that of FCM with squared Euclidean distance on real world and synthetic datasets (even in absence of the noise features).

[1]  Abdolhossein Sarrafzadeh,et al.  Fuzzy C-means based on Automated Variable Feature Weighting , 2013 .

[2]  Ka Yee Yeung,et al.  Details of the Adjusted Rand index and Clustering algorithms Supplement to the paper “ An empirical study on Principal Component Analysis for clustering gene expression data ” ( to appear in Bioinformatics ) , 2001 .

[3]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[4]  Pasi Fränti,et al.  Iterative shrinking method for clustering problems , 2006, Pattern Recognit..

[5]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[7]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[8]  Sanghamitra Bandyopadhyay,et al.  Classification and learning using genetic algorithms - applications in bioinformatics and web intelligence , 2007, Natural computing series.

[9]  Marc Teboulle,et al.  Clustering with Entropy-Like k-Means Algorithms , 2006, Grouping Multidimensional Data.

[10]  Sadaaki Miyamoto,et al.  Algorithms for Fuzzy Clustering - Methods in c-Means Clustering with Applications , 2008, Studies in Fuzziness and Soft Computing.

[11]  Ujjwal Maulik,et al.  Genetic clustering for automatic evolution of clusters and application to image classification , 2002, Pattern Recognit..

[12]  Hsiang-Chuan Liu,et al.  Fuzzy C-Mean Algorithm Based on Mahalanobis Distances and Better Initial Values , 2007 .

[13]  Marc Teboulle,et al.  A Unified Continuous Optimization Framework for Center-Based Clustering Methods , 2007, J. Mach. Learn. Res..

[14]  Jongwoo Kim,et al.  A note on the Gustafson-Kessel and adaptive fuzzy clustering algorithms , 1999, IEEE Trans. Fuzzy Syst..

[15]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[16]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[17]  Cor J. Veenman,et al.  A Maximum Variance Cluster Algorithm , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Hui Xiong,et al.  A Generalization of Distance Functions for Fuzzy $c$ -Means Clustering With Centroids of Arithmetic Means , 2012, IEEE Transactions on Fuzzy Systems.

[19]  M. Cugmas,et al.  On comparing partitions , 2015 .

[20]  Ka Yee Yeung,et al.  Principal component analysis for clustering gene expression data , 2001, Bioinform..

[21]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[22]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[23]  James C. Bezdek,et al.  Convergence of Alternating Optimization , 2003, Neural Parallel Sci. Comput..