Stronger Convergence Results for the Center-Based Fuzzy Clustering With Convex Divergence Measure

We present a novel alternative convergence theory of the fuzzy $C$ -means (FCM) clustering algorithm with a super-class of the so-called “distance like functions” which emerged from the earlier attempts of unifying the theories of center-based clustering methods. This super-class does not assume the existence of double derivative of the distance measure with respect to the coordinate of the cluster representative (first coordinate in this formulation). The convergence result does not require the separability of the distance measures. Moreover, it provides us with a stronger convergence property comparable (same to be precise, but in terms of the generalized distance measure) to that of the classical FCM with squared Euclidean distance. The crux of the convergence analysis lies in the development of a fundamentally novel mathematical proof of the continuity of the clustering operator even in absence of the closed form upgrading rule, without necessitating the separability and double differentiability of the distance function and still providing us with a convergence result comparable to that of the classical FCM. The implication of our novel proof technique goes way beyond the realm of FCM and provides a general setup for convergence analysis of the similar iterative algorithms.

[1]  Biao Hou,et al.  Discriminative Transformation Learning for Fuzzy Sparse Subspace Clustering , 2017, IEEE Transactions on Cybernetics.

[2]  Lawrence O. Hall,et al.  Convergence of the Single-Pass and Online Fuzzy C-Means Algorithms , 2011, IEEE Transactions on Fuzzy Systems.

[3]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[4]  James C. Bezdek,et al.  Optimization of clustering criteria by reformulation , 1995, IEEE Trans. Fuzzy Syst..

[5]  Jongwoo Kim,et al.  A note on the Gustafson-Kessel and adaptive fuzzy clustering algorithms , 1999, IEEE Trans. Fuzzy Syst..

[6]  Rui Xu,et al.  Survey of clustering algorithms , 2005, IEEE Transactions on Neural Networks.

[7]  John F. Kolen,et al.  Reducing the time complexity of the fuzzy c-means algorithm , 2002, IEEE Trans. Fuzzy Syst..

[8]  Swagatam Das,et al.  Optimizing cluster structures with inner product induced norm based dissimilarity measures: Theoretical development and convergence analysis , 2016, Inf. Sci..

[9]  Swagatam Das,et al.  Geometric divergence based fuzzy clustering with strong resilience to noise features , 2016, Pattern Recognit. Lett..

[10]  Swagatam Das,et al.  On the unification of possibilistic fuzzy clustering: Axiomatic development and convergence analysis , 2017, Fuzzy Sets Syst..

[11]  Swagatam Das,et al.  Feature-weighted clustering with inner product induced norm based dissimilarity measures: an optimization perspective , 2017, Machine Learning.

[12]  Yingjie Tian,et al.  A Comprehensive Survey of Clustering Algorithms , 2015, Annals of Data Science.

[13]  L. Hubert,et al.  Comparing partitions , 1985 .

[14]  Korris Fu-Lai Chung,et al.  Generalized Fuzzy C-Means Clustering Algorithm With Improved Fuzzy Partitions , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[16]  Jens Jäkel,et al.  A New Convergence Proof of Fuzzy c-Means , 2005, IEEE Trans. Fuzzy Syst..

[17]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[18]  James C. Bezdek,et al.  Local convergence of the fuzzy c-Means algorithms , 1986, Pattern Recognit..

[19]  Hui Xiong,et al.  A Generalization of Distance Functions for Fuzzy $c$ -Means Clustering With Centroids of Arithmetic Means , 2012, IEEE Transactions on Fuzzy Systems.

[20]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[21]  Paulo Fazendeiro,et al.  Observer-Biased Fuzzy Clustering , 2015, IEEE Transactions on Fuzzy Systems.

[22]  Swagatam Das,et al.  k-Means clustering with a new divergence-based distance metric: Convergence and performance analysis , 2017, Pattern Recognit. Lett..

[23]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Teh Ying Wah,et al.  A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data , 2015, PloS one.

[25]  Tamalika Chaira,et al.  A novel intuitionistic fuzzy C means clustering algorithm and its application to medical images , 2011, Appl. Soft Comput..

[26]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[27]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Scalable Comput. Pract. Exp..

[28]  Y. Censor,et al.  An iterative row-action method for interval convex programming , 1981 .

[29]  James C. Bezdek,et al.  Convergence of Alternating Optimization , 2003, Neural Parallel Sci. Comput..

[30]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[31]  Jian Yu,et al.  Analysis of Parameter Selection for Gustafson–Kessel Fuzzy Clustering Using Jacobian Matrix , 2015, IEEE Transactions on Fuzzy Systems.

[32]  A. Barron,et al.  APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[33]  Jian Yu,et al.  Optimality test for generalized FCM and its application to parameter selection , 2005, IEEE Transactions on Fuzzy Systems.

[34]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[35]  Kuo-Ping Lin,et al.  A Novel Evolutionary Kernel Intuitionistic Fuzzy $C$ -means Clustering Algorithm , 2014, IEEE Transactions on Fuzzy Systems.

[36]  Marc Teboulle,et al.  A Unified Continuous Optimization Framework for Center-Based Clustering Methods , 2007, J. Mach. Learn. Res..

[37]  Shokri Z. Selim,et al.  On the Local Optimality of the Fuzzy Isodata Clustering Algorithm , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  W. T. Tucker,et al.  Convergence theory for fuzzy c-means: Counterexamples and repairs , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[40]  James C. Bezdek,et al.  Optimality tests for fixed points of the fuzzy c-means algorithm , 1988, Pattern Recognit..

[41]  L. Bregman The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[42]  I. Csiszár $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[43]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Hsiang-Chuan Liu,et al.  Fuzzy C-Mean Algorithm Based on Mahalanobis Distances and Better Initial Values , 2007 .

[45]  Jian Yu,et al.  Analysis of the weighting exponent in the FCM , 2004, IEEE Trans. Syst. Man Cybern. Part B.