On the unification of possibilistic fuzzy clustering: Axiomatic development and convergence analysis

Abstract Fuzzy partitional clustering algorithms like Fuzzy C-means (FCM) are sensitive to noise points or outliers because of the probabilistic constraint (the sum of the membership values of a specific data point to all the clusters is 1), which enables it to classify any noise or outlier in a specific cluster. This, not only hampers the clustering performance corresponding to that specific point, but also leaves a huge impact on the overall clustering process by deviating the cluster centroids through a significant amount. In order to improve this weakness, possibilistic approach relaxes the probabilistic constraints. Nevertheless, due to lack of constraints imposed on the typicality matrix, depending on initialization, possibilistic clustering algorithms suffer from coincident clusters because their membership expressions do not consider the distance to other cluster representatives. Fuzzy possibilistic clustering which puts a linear constraint on the sum of all the typicality values, takes care of this problem, but if the number of data points is high, the generated typicality values are very small due to the linear constraint on their sums. In order to do away with that problem, the possibilistic fuzzy clustering algorithm was developed. In this article, we present an axiomatic development and extension of the possibilistic fuzzy clustering algorithm in three directions: choice of the dissimilarity measure, joint contribution function, and the penalty function. We provide a thorough convergence analysis of the proposed generalized possibilistic fuzzy clustering algorithm. We investigate the relationships of the proposed generalization with the existing variations of the Possibilistic C-Means (PCM), Fuzzy Possiblistic C-Means (FPCM) and Possibilistic Fuzzy C-Means (PFCM) algorithms in literature. To the best of our knowledge, this is the first article of its kind to provide a unification to the long list of possibilistic, fuzzy possiblistic, and possibilistic fuzzy clustering methods.

[1]  James M. Keller,et al.  The possibilistic C-means algorithm: insights and recommendations , 1996, IEEE Trans. Fuzzy Syst..

[2]  Swagatam Das,et al.  Geometric divergence based fuzzy clustering with strong resilience to noise features , 2016, Pattern Recognit. Lett..

[3]  J. Bezdek,et al.  c-means clustering with the l/sub l/ and l/sub infinity / norms , 1991 .

[4]  James C. Bezdek,et al.  A mixed c-means clustering model , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[5]  Frank Klawonn,et al.  An alternative approach to the fuzzifier in fuzzy clustering to obtain better clustering , 2003, EUSFLAT Conf..

[6]  Y. Censor,et al.  Parallel Optimization: Theory, Algorithms, and Applications , 1997 .

[7]  Y. Censor,et al.  An iterative row-action method for interval convex programming , 1981 .

[8]  Frank Hoeppner,et al.  Fuzzy shell clustering algorithms in image processing: fuzzy C-rectangular and 2-rectangular shells , 1997, IEEE Trans. Fuzzy Syst..

[9]  Frank Klawonn,et al.  A contribution to convergence theory of fuzzy c-means and derivatives , 2003, IEEE Trans. Fuzzy Syst..

[10]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Hui Xiong,et al.  A Generalization of Distance Functions for Fuzzy $c$ -Means Clustering With Centroids of Arithmetic Means , 2012, IEEE Transactions on Fuzzy Systems.

[12]  Kuo-Lung Wu,et al.  Unsupervised possibilistic clustering , 2006, Pattern Recognit..

[13]  Frank Klawonn,et al.  Fuzzy Clustering Based on Modified Distance Measures , 1999, IDA.

[14]  Marc Teboulle,et al.  A Unified Continuous Optimization Framework for Center-Based Clustering Methods , 2007, J. Mach. Learn. Res..

[15]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[16]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[17]  Hichem Frigui,et al.  Fuzzy and possibilistic shell clustering algorithms and their application to boundary detection and surface approximation. I , 1995, IEEE Trans. Fuzzy Syst..

[18]  L. Bregman.  The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming , 1967 .

[19]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[20]  S. Miyamoto,et al.  Algorithms for L1 and Lp Fuzzy c-Means and Their Convergence , 1998 .

[21]  Frank Klawonn,et al.  What Is Fuzzy about Fuzzy Clustering? Understanding and Improving the Concept of the Fuzzifier , 2003, IDA.

[22]  James C. Bezdek,et al.  Generalized fuzzy c-means clustering strategies using Lp norm distances , 2000, IEEE Trans. Fuzzy Syst..

[23]  James M. Keller,et al.  A possibilistic fuzzy c-means clustering algorithm , 2005, IEEE Transactions on Fuzzy Systems.

[24]  C. Borgelt.  Objective Functions for Fuzzy Clustering , 2013 .

[25]  I. Csiszár.  $I$-Divergence Geometry of Probability Distributions and Minimization Problems , 1975 .

[26]  Sadaaki Miyamoto,et al.  Sparse Possibilistic Clustering with L1 Regularization , 2007 .

[27]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[28]  Andreas Nürnberger,et al.  Computational Intelligence in Intelligent Data Analysis , 2013, Studies in Computational Intelligence.

[29]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[30]  Swagatam Das,et al.  Axiomatic generalization of the membership degree weighting function for fuzzy C means clustering: Theoretical development and convergence analysis , 2017, Inf. Sci..