Processing images by semi-linear predictability minimization.

In the predictability minimization approach, input patterns are fed into a system consisting of adaptive, initially unstructured feature detectors. There are also adaptive predictors constantly trying to predict current feature detector outputs from other feature detector outputs. Simultaneously, however, the feature detectors try to become as unpredictable as possible, resulting in a co-evolution of predictors and feature detectors. This paper describes the implementation of a visual processing system trained by semi-linear predictability minimization, and presents many experiments that examine its response to artificial and real-world images. In particular, we observe that under a wide variety of conditions, predictability minimization results in the development of well-known visual feature detectors.

[1]  Jürgen Schmidhuber,et al.  Discovering Predictable Classifications , 1993, Neural Computation.

[2]  Lucas C. Parra,et al.  Statistical Independence and Novelty Detection with Information Preserving Nonlinear Maps , 1996, Neural Computation.

[3]  J. Rubner,et al.  Development of feature detectors by self-organization. A network model. , 1990, Biological cybernetics.

[4]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[5]  Néstor Parga,et al.  Redundancy Reduction and Independent Component Analysis: Conditions on Cumulants and Adaptive Approaches , 1997, Neural Computation.

[6]  J. Urgen Schmidhuber.  Learning Factorial Codes by Predictability Minimization , 1992 .

[7]  Jürgen Schmidhuber,et al.  Netzwerkarchitekturen, Zielfunktionen und Kettenregel , 1993 .

[8]  Nicole Norbert Schraudolph.  Optimization of entropy with neural networks , 1996 .

[9]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[10]  Lucas C. Parra,et al.  Non-linear Feature Extraction by Redundancy Reduction in an Unsupervised Stochastic Neural Network , 1997, Neural Networks.

[11]  KD Miller.  A model for the development of simple cell receptive fields and the ordered arrangement of orientation columns through activity-dependent competition between ON- and OFF-center inputs , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  J. Rubner,et al.  A self-organizing network for principal-component analysis , 1989 .

[13]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[14]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[15]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[16]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[17]  H. B. Barlow,et al.  Finding Minimum Entropy Codes , 1989, Neural Computation.

[18]  Jürgen Schmidhuber,et al.  Semilinear Predictability Minimization Produces Well-Known Feature Detectors , 1996, Neural Computation.

[19]  Zhaoping Li,et al.  Understanding Retinal Color Coding from First Principles , 1992, Neural Computation.

[20]  Terrence J. Sejnowski,et al.  Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain , 1992, NIPS.

[21]  G. Cottrell.  Optimization of Entropy with Neural Networks , 1995 .

[22]  ParraLucas,et al.  Statistical independence and novelty detection with information preserving nonlinear maps , 1996 .

[23]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[24]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[25]  Jürgen Schmidhuber,et al.  Feature Extraction Through LOCOCODE , 1999, Neural Computation.

[26]  Nicol N. Schraudolph,et al.  Local Gain Adaptation in Stochastic Gradient Descent , 1999 .

[27]  Stefanie N. Lindstaedt,et al.  Comparison of two Unsupervised Neural Network Models for Redundancy Reduction , 1993 .

[28]  Corso Elvezia.  Neural Predictors for Detecting and Removing Redundant Information , 1998 .