Maximally fault tolerant neural networks

An application of neural network modeling is described for generating hypotheses about the relationships between response properties of neurons and information processing in the auditory system. The goal is to study response properties that are useful for extracting sound localization information from directionally selective spectral filtering provided by the pinna. For studying sound localization based on spectral cues provided by the pinna, a feedforward neural network model with a guaranteed level of fault tolerance is introduced. Fault tolerance and uniform fault tolerance in a neural network are formally defined and a method is described to ensure that the estimated network exhibits fault tolerance. The problem of estimating weights for such a network is formulated as a large-scale nonlinear optimization problem. Numerical experiments indicate that solutions with uniform fault tolerance exist for the pattern recognition problem considered. Solutions derived by introducing fault tolerance constraints have better generalization properties than solutions obtained via unconstrained back-propagation.

[1]  David J. Burr,et al.  Experiments on neural net recognition of spoken and written text , 1988, IEEE Trans. Acoust. Speech Signal Process..

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  William M. Jenkins,et al.  Effects of unilateral lesions in the central auditory system on sound localization behavior , 1981 .

[4]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[5]  Terrence J. Sejnowski,et al.  NETtalk: a parallel network that learns to read aloud , 1988 .

[6]  T Poggio,et al.  Regularization Algorithms for Learning That Are Equivalent to Multilayer Networks , 1990, Science.

[7]  M. Liberman The cochlear frequency map for the cat: labeling auditory-nerve fibers of known characteristic frequency. , 1982, The Journal of the Acoustical Society of America.

[8]  W. Jenkins,et al.  Sound localization: effects of unilateral lesions in central auditory system. , 1982, Journal of neurophysiology.

[9]  Terrence J. Sejnowski,et al.  Network model of shape-from-shading: neural function arises from both receptive and projective fields , 1988, Nature.

[10]  W. R. Webster,et al.  Discharge patterns of single units in inferior colliculus of the alert cat. , 1972, Journal of neurophysiology.

[11]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[12]  D. Irvine,et al.  Topographic organization of interaural intensity difference sensitivity in deep layers of cat superior colliculus: implications for auditory spatial representation. , 1985, Journal of neurophysiology.

[13]  E D Young,et al.  Organization of dorsal cochlear nucleus type IV unit response maps and their relationship to activation by bandlimited noise. , 1991, Journal of neurophysiology.

[14]  Bradley W. Dickinson,et al.  Trellis codes, receptive fields, and fault tolerant, self-repairing neural networks , 1990, IEEE Trans. Neural Networks.

[15]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[16]  P. Grobstein,et al.  Neuronal organization underlying visually elicited prey orienting in the frog—II. Anatomical studies on the laterality of central projections , 1987, Neuroscience.

[17]  Richard A. Andersen,et al.  A back-propagation programmed network that simulates response properties of a subset of posterior parietal neurons , 1988, Nature.

[18]  E. Young,et al.  Responses to tones and noise of single cells in dorsal cochlear nucleus of unanesthetized cats. , 1976, Journal of neurophysiology.

[19]  John J. Rice,et al.  Directional dependence of free‐field‐to‐eardum transformations in cats , 1989 .

[20]  Terrence J. Sejnowski,et al.  Learned classification of sonar targets using a massively parallel network , 1988, IEEE Trans. Acoust. Speech Signal Process..

[21]  Eric L. Schwartz,et al.  Computational Neuroscience , 1993, Neuromethods.

[22]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[23]  W. D. Neff,et al.  Auditory localization: role of auditory pathways in brain stem of the cat. , 1975, Journal of neurophysiology.

[24]  Michael C. Mozer,et al.  Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[25]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[26]  J. E. Hind,et al.  Direction-dependent spectral properties of cat external ear: new data and cross-species comparisons. , 1990, The Journal of the Acoustical Society of America.

[27]  J. C. Middlebrooks,et al.  A neural code for auditory space in the cat's superior colliculus , 1984, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[28]  J. Hebrank,et al.  Spectral cues used in the localization of sound sources on the median plane. , 1974, The Journal of the Acoustical Society of America.

[29]  Hervé Bourlard,et al.  Generalization and Parameter Estimation in Feedforward Netws: Some Experiments , 1989, NIPS.