Robust growing neural gas algorithm with application in cluster analysis

We propose a novel robust clustering algorithm within the Growing Neural Gas (GNG) framework, called Robust Growing Neural Gas (RGNG) network.The Matlab codes are available from . By incorporating several robust strategies, such as outlier resistant scheme, adaptive modulation of learning rates and cluster repulsion method into the traditional GNG framework, the proposed RGNG network possesses better robustness properties. The RGNG is insensitive to initialization, input sequence ordering and the presence of outliers. Furthermore, the RGNG network can automatically determine the optimal number of clusters by seeking the extreme value of the Minimum Description Length (MDL) measure during network growing process. The resulting center positions of the optimal number of clusters represented by prototype vectors are close to the actual ones irrespective of the existence of outliers. Topology relationships among these prototypes can also be established. Experimental results have shown the superior performance of our proposed method over the original GNG incorporating MDL method, called GNG-M, in static data clustering tasks on both artificial and UCI data sets.

[1]  Douglas A. Baxter,et al.  Small networks of empirically derived adaptive elements simulate some higher-order features of classical conditioning , 1990, Neural Networks.

[2]  M.C. Clark,et al.  MRI segmentation using fuzzy clustering techniques , 1994, IEEE Engineering in Medicine and Biology Magazine.

[3]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[4]  T. Martínez,et al.  Competitive Hebbian Learning Rule Forms Perfectly Topology Preserving Maps , 1993 .

[5]  Ponnuthurai N. Suganthan,et al.  Boosting the HONG network , 2003, Neurocomputing.

[6]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[7]  Jitender S. Deogun,et al.  Conceptual clustering in information retrieval , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[8]  Pavel Berkhin,et al.  A Survey of Clustering Data Mining Techniques , 2006, Grouping Multidimensional Data.

[9]  James C. Bezdek,et al.  A mixed c-means clustering model , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[10]  Giulio Sandini,et al.  An incremental growing neural network and its application to robot control , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[11]  Hichem Frigui,et al.  A Robust Competitive Clustering Algorithm With Applications in Computer Vision , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  P.N. Suganthan,et al.  A Robust Neural Gas algorithm for clustering analysis , 2004, International Conference on Intelligent Sensing and Information Processing, 2004. Proceedings of.

[13]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[14]  Hichem Frigui,et al.  Clustering by competitive agglomeration , 1997, Pattern Recognit..

[15]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[16]  Mineichi Kudo,et al.  MDL-Based Selection of the Number of Components in Mixture Models for Pattern Classification , 1998, SSPR/SPR.

[17]  Stanley C. Ahalt,et al.  Competitive learning algorithms for vector quantization , 1990, Neural Networks.

[18]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[19]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[20]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[21]  Siddheswar Ray,et al.  Determination of Number of Clusters in K-Means Clustering and Application in Colour Image Segmentation , 2000 .

[22]  Giulio Sandini,et al.  Neural-gas for function approximation: a heuristic for minimizing the local estimation error , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[23]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[24]  Moshe Kam,et al.  The Credibilistic Fuzzy C Means Algorithm , 1998 .

[25]  Rajesh N. Davé,et al.  Robust clustering methods: a unified view , 1997, IEEE Trans. Fuzzy Syst..

[26]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[27]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[28]  B Fritzke,et al.  A growing neural gas network learns topologies. G. Tesauro, DS Touretzky, and TK Leen, editors , 1995, NIPS 1995.

[29]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[30]  Greg Hamerly,et al.  Learning the k in k-means , 2003, NIPS.

[31]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[32]  Habtom W. Ressom,et al.  Adaptive double self-organizing maps for clustering gene expression profiles , 2003, Neural Networks.

[33]  Xiang Cao,et al.  Video shot motion characterization based on hierarchical overlapped growing neural gas networks , 2003, Multimedia Systems.

[34]  R. Zemel.  A minimum description length framework for unsupervised learning , 1994 .

[35]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[36]  Miin-Shen Yang,et al.  Alternative c-means clustering algorithms , 2002, Pattern Recognit..