Detecting the Number of Clusters in n-Way Probabilistic Clustering

Recently, there has been a growing interest in multiway probabilistic clustering. Some efficient algorithms have been developed for this problem. However, not much attention has been paid on how to detect the number of clusters for the general n-way clustering (n ≥ 2). To fill this gap, this problem is investigated based on n-way algebraic theory in this paper. A simple, yet efficient, detection method is proposed by eigenvalue decomposition (EVD), which is easy to implement. We justify this method. In addition, its effectiveness is demonstrated by the experiments on both simulated and real-world data sets.

[1]  H. Kiers,et al.  Three-mode principal components analysis: choosing the numbers of components and sensitivity to local optima. , 2000, The British journal of mathematical and statistical psychology.

[2]  H. Bozdogan Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[3]  Jorma Rissanen,et al.  The Minimum Description Length Principle in Coding and Modeling , 1998, IEEE Trans. Inf. Theory.

[4]  Stanley L. Sclove,et al.  Some Aspects of Model-Selection Criteria , 1994 .

[5]  Pascal Larzabal,et al.  Some properties of ordered eigenvalues of a Wishart matrix: application in detection test and model order selection , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[6]  Joachim M. Buhmann,et al.  A maximum entropy approach to pairwise data clustering , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[7]  Robert Tibshirani,et al.  Estimating the number of clusters in a data set via the gap statistic , 2000 .

[8]  Amnon Shashua,et al.  A unifying approach to hard and probabilistic clustering , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  A. Cuevas,et al.  Estimating the number of clusters , 2000 .

[10]  R. Bro PARAFAC. Tutorial and applications , 1997 .

[11]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[12]  André Hardy,et al.  An examination of procedures for determining the number of clusters in a data set , 1994 .

[13]  M. Haardt,et al.  Robust methods based on the hosvd for estimating the model order in PARAFAC models , 2008, 2008 5th IEEE Sensor Array and Multichannel Signal Processing Workshop.

[14]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[15]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[17]  Arindam Banerjee,et al.  Multi-way Clustering on Relation Graphs , 2007, SDM.

[18]  Lei Xu,et al.  Investigation on Several Model Selection Criteria for Determining the Number of Cluster , 2004 .

[19]  J. Kruskal Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[20]  Pieter M. Kroonenberg,et al.  Multiplicatieve decompositie van interacties bij oordelen over de werkelijkheidswaarde van televisiefilms [Multiplicative decomposition of interactions for judgements of realism of television films] , 1987 .

[21]  R. Harshman The differences between analysis of covariance and correlation , 2001 .

[22]  M. Haardt,et al.  Enhanced Model Order Estimation using Higher-Order Arrays , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[23]  Pierre Comon,et al.  Generic and typical ranks of three-way arrays , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[24]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[25]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[26]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.

[27]  W. Krzanowski,et al.  A Criterion for Determining the Number of Groups in a Data Set Using Sum-of-Squares Clustering , 1988 .

[28]  R. Bro,et al.  A new efficient method for determining the number of components in PARAFAC models , 2003 .

[29]  Henri Maître,et al.  Kernel MDL to Determine the Number of Clusters , 2007, MLDM.

[30]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[31]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[32]  Keying Ye,et al.  Determining the Number of Clusters Using the Weighted Gap Statistic , 2007, Biometrics.

[33]  Rasmus Bro,et al.  The N-way Toolbox for MATLAB , 2000 .

[34]  Richard A. Harshman,et al.  Determination and Proof of Minimum Uniqueness Conditions for PARAFAC1 , 1972 .

[35]  André Quinquis,et al.  A New Method for Estimating the Number of Harmonic Components in Noise with Application in High Resolution Radar , 2004, EURASIP J. Adv. Signal Process..

[36]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Victor Solo,et al.  Dimension Estimation in Noisy PCA With SURE and Random Matrix Theory , 2008, IEEE Transactions on Signal Processing.

[38]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[39]  E Donchin,et al.  The mental prosthesis: assessing the speed of a P300-based brain-computer interface. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[40]  C. L. Philip Chen,et al.  Cluster number selection for a small set of samples using the Bayesian Ying-Yang model , 2002, IEEE Trans. Neural Networks.

[41]  Amnon Shashua,et al.  Doubly Stochastic Normalization for Spectral Clustering , 2006, NIPS.

[42]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[43]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[44]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[46]  Charles M. Grinstead,et al.  Introduction to probability , 1999, Statistics for the Behavioural Sciences.

[47]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[48]  Johan Håstad,et al.  Tensor Rank is NP-Complete , 1989, ICALP.

[49]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[50]  atherine,et al.  Finding the number of clusters in a data set : An information theoretic approach C , 2003 .

[51]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[52]  Henk A L Kiers,et al.  A fast method for choosing the numbers of components in Tucker3 analysis. , 2003, The British journal of mathematical and statistical psychology.

[53]  Tamir Hazan,et al.  Multi-way Clustering Using Super-Symmetric Non-negative Tensor Factorization , 2006, ECCV.

[54]  Martin Haardt,et al.  Model Order Selection for Short Data: An Exponential Fitting Test (EFT) , 2007, EURASIP J. Adv. Signal Process..

[55]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[56]  H. Akaike A new look at the statistical model identification , 1974 .

[57]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[58]  Tom Minka,et al.  Automatic Choice of Dimensionality for PCA , 2000, NIPS.

[59]  F. Piccione,et al.  P300-based brain computer interface: Reliability and performance in healthy and paralysed participants , 2006, Clinical Neurophysiology.

[60]  H. Kiers,et al.  Selecting among three-mode principal component models of different types and complexities: a numerical convex hull based method. , 2006, The British journal of mathematical and statistical psychology.

[61]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .