UNIQUENESS OF WEIGHTS FOR NEURAL NETWORKS

of the weights for minimal feedforward nets with a given input-output map," Neural Networks 5(1992): 589-593. 13 2'. for each i 6 = j, there exists some k such that b i;k 6 = b j;k , and dening ~ S as above, but with this new B n;m , the above reference also proved: Theorem 3 Assume that is not odd and satises property (*). For any two 6; ^ 6, 6 ^ 6 if and only if 6 and ^ 6 are permutation equivalent. The paper [1] explains how in fact the assumption that both nets have the same activation function is basically redundant, as the equality of activation functions can be derived from the equality of behaviors. Many more results are given there, for other continuous-time models. 3.2 Discrete-Time Similar results hold for discrete-time recurrent nets. These are treated in detail in [2]. Proofs are technically dierent than in the continuous case, but the results are analogous. More precisely, we assume that not only satises (*), but also the following extra condition, which appeared above in the context of single-hidden layer nets with no osets: (k) (0) 6 = 0 for innitely many integers k. Then the same theorems as before hold, provided that we redene: We assume from now on that is innitely dierentiable, and that it satises the following assumptions:

[1]  M. Newman,et al.  Interpolation and approximation , 1965 .

[2]  W. Rugh,et al.  On the structure of a class of nonlinear systems , 1973, CDC 1973.

[3]  W. Rugh,et al.  Structural features of factorable Volterra systems , 1976 .

[4]  Stephen P. Boyd,et al.  Uniqueness of a basic nonlinear structure , 1983 .

[5]  P. Diaconis,et al.  On Nonlinear Functions of Linear Combinations , 1984 .

[6]  Lawrence D. Jackel,et al.  Large Automatic Learning, Rule Extraction, and Generalization , 1987, Complex Syst..

[7]  Eduardo D. Sontag,et al.  Backpropagation separates when perceptrons do , 1989, International 1989 Joint Conference on Neural Networks.

[8]  Eduardo D. Sontag,et al.  Backpropagation Can Give Rise to Spurious Local Minima Even for Networks without Hidden Layers , 1989, Complex Syst..

[9]  J. Slawny,et al.  Back propagation fails to separate where perceptrons succeed , 1989 .

[10]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[11]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[12]  Eduardo Sontag,et al.  For neural networks, function determines form , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[13]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[14]  Eduardo D. Sontag,et al.  Identi ability of discrete-time neural networks , 1993 .

[15]  Uwe Helmke,et al.  Existence and uniqueness results for neural network approximations , 1995, IEEE Trans. Neural Networks.

[16]  Stephen P. Boyd,et al.  Uniqueness of circuits and systems containing one nonlinearity , 1985 .