Visual feature analysis by the self-organising maps

The Self-Organising Map (SOM) is an Artificial Neural Network (ANN) model consisting of a regular grid of processing units. A model of some multidimensional observation, e.g. a class of digital images, is associated with each unit. The map attempts to represent all the available observations using a restricted set of models. In unsupervised learning, the models become ordered on the grid so that similar models are close to each other. We review here the objective functions and learning rules related to the SOM, starting from vector coding based on a Euclidean metric and extending the theory of arbitrary metrics and to a subspace formalism, in which each SOM unit represents a subspace of the observation space. It is shown that this Adaptive-Subspace SOM (ASSOM) is able to create sets of wavelet- and Gabor-type filters when randomly displaced or moving input patterns are used as training data. No analytical functional form for these filters is thereby postulated. The same kind of adaptive system can create many other kinds of invariant visual filters, like rotation or scale-invariant filters, if there exist corresponding transformations in the training data. The ASSOM system can act as a learning feature-extraction stage for pattern recognisers, being able to adapt to arbitrary sensory environments. We then show that the invariant Gabor features can be effectively used in face recognition, whereby the sets of Gabor filter outputs are coded with the SOM and a face is represented by the histogram over the SOM units.

[1]  Dennis Gabor,et al.  Theory of communication , 1946 .

[2]  Ronen Basri,et al.  Recognition by Linear Combinations of Models , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  C. von der Malsburg,et al.  Distortion invariant object recognition by matching hierarchically labeled graphs , 1989, International 1989 Joint Conference on Neural Networks.

[4]  J. Daugman.  Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[5]  S. Zeki.  The representation of colours in the cerebral cortex , 1980, Nature.

[6]  N. Suga,et al.  Neural axis representing target range in the auditory cortex of the mustache bat. , 1979, Science.

[7]  Teuvo Kohonen,et al.  Physiological interpretationm of the self-organizing map algorithm , 1993 .

[8]  R. Didday.  A model of visuomotor mechanisms in the frog optic tectum , 1976 .

[9]  C. Lee Giles,et al.  Encoding Geometric Invariances in Higher-Order Neural Networks , 1987, NIPS.

[10]  Stephen Grossberg,et al.  Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions , 1976, Biological Cybernetics.

[11]  Joachim M. Buhmann,et al.  Size and distortion invariant object recognition by hierarchical graph matching , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[12]  Samuel Kaski,et al.  Self-Organized Formation of Various Invariant-Feature Filters in the Adaptive-Subspace SOM , 1997, Neural Computation.

[13]  Teuvo Kohonen,et al.  Physiological interpretation of the Self-Organizing Map algorithm , 1993, Neural Networks.

[14]  J. Lampinen,et al.  Unsupervised Learning for Feature Extraction , 1994 .

[15]  S. T. Toborg,et al.  An approach to image recognition using sparse filter graphs , 1989, International 1989 Joint Conference on Neural Networks.

[16]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[17]  T. Kohonen,et al.  The subspace learning algorithm as a formalism for pattern recognition and neural networks , 1988, IEEE 1988 International Conference on Neural Networks.

[18]  G. Granlund.  In search of a general picture processing operator , 1978 .

[19]  Allen Gersho,et al.  Asymptotically optimal block quantization , 1979, IEEE Trans. Inf. Theory.

[20]  S. Grossberg,et al.  Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors , 1976, Biological Cybernetics.

[21]  John G. Daugman,et al.  Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression , 1988, IEEE Trans. Acoust. Speech Signal Process..

[22]  Jouko Lampinen,et al.  Distortion tolerant pattern recognition based on self-organizing feature extraction , 1995, IEEE Trans. Neural Networks.

[23]  Samuel Kaski,et al.  Winner-take-all networks for physiological models of competitive learning , 1994, Neural Networks.

[24]  Stephen Coombes,et al.  Learning higher order correlations , 1993, Neural Networks.

[25]  Risto Miikkulainen,et al.  Cooperative self-organization of afferent and lateral connections in cortical maps , 1994, Biological Cybernetics.

[26]  T. Kohonen.  SELF-ORGANIZING MAPS: OPHMIZATION APPROACHES , 1991 .

[27]  S. Amari,et al.  Competition and Cooperation in Neural Nets , 1982 .

[28]  J. Makhoul,et al.  Vector quantization in speech coding , 1985, Proceedings of the IEEE.

[29]  Fabio Cocurullo,et al.  A new algorithm for vector quantization , 1995, Proceedings DCC '95 Data Compression Conference.

[30]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[31]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[32]  L. Rabiner,et al.  The acoustics, speech, and signal processing society - A historical perspective , 1984, IEEE ASSP Magazine.

[33]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[34]  Paulo J. G. Lisboa,et al.  Translation, rotation, and scale invariant pattern recognition by high-order neural networks and moment classifiers , 1992, IEEE Trans. Neural Networks.

[35]  R. Gray,et al.  Vector quantization , 1984, IEEE ASSP Magazine.

[36]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[37]  Teuvo Kohonen,et al.  Emergence of invariant-feature detectors in the adaptive-subspace self-organizing map , 1996, Biological Cybernetics.