On quantization error of self-organizing map network

Abstract In this paper, we analyze how neighborhood size and number of weights in the self-organizing map (SOM) effect quantization error. A sequence of i.i.d. one-dimensional random variable with uniform distribution is considered as input of the SOM. First obtained is the linear equation that an equilibrium state of the SOM satisfies with any neighborhood size and number of weights. Then it is shown that the SOM converges to the unique minimum point of quantization error if and only if the neighborhood size is one, the smallest. If the neighborhood size increases with the increasing number of weights at the same ratio, the asymptotic quantization error does not converge to zero and the asymptotic distribution of weights differs from the distribution of input samples. This suggests that in order to achieve a small quantization error and good approximation of input distribution, a small neighborhood size must be used. Weight distributions in numerical evaluation confirm the result.

[1]  J. Makhoul,et al.  Vector quantization in speech coding , 1985, Proceedings of the IEEE.

[2]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[3]  Hujun Yin,et al.  On the Distribution and Convergence of Feature Space in Self-Organizing Maps , 1995, Neural Computation.

[4]  Han-fu Chʿen Recursive estimation and control for stochastic systems , 1985 .

[5]  John G. Taylor,et al.  On the Ordering Conditions for Self-Organizing Maps , 1995, Neural Computation.

[6]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[7]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[8]  Paul L. Zador,et al.  Asymptotic quantization error of continuous signals and the quantization dimension , 1982, IEEE Trans. Inf. Theory.

[9]  Gilles Pagès,et al.  Convergence in Distribution of the One-Dimensional Kohonen Algorithms when the Stimuli are not Uniform , 1994, Advances in Applied Probability.

[10]  Zhen-Ping Lo,et al.  Analysis of the convergence properties of topology preserving neural networks , 1993, IEEE Trans. Neural Networks.

[11]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[12]  Helge Ritter Asymptotic level density for a class of vector quantization processes , 1991, IEEE Trans. Neural Networks.

[13]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[14]  G. Pagès,et al.  Self-organization and a.s. convergence of the one-dimensional Kohonen algorithm with non-uniformly distributed stimuli , 1993 .