For neural networks, function determines form

It is proved that, generically on nets, the I/O (input-output) behavior uniquely determines the internal form, up to simple symmetries. The sets where this conclusion does not hold are thin in the sense that they are included in sets defined by algebraic equalities. It is shown that, under very weak genericity assumptions, the following is true: assume given two nets, whose neurons all have the same nonlinear activation function sigma ; if the two sets have equal behaviors as 'black boxes', then necessarily they must have the same number of neurons and, except at most for sign reversals at each node, the same weights. The results obtained imply unique identifiability of parameters, under all possible I/O experiments. It is also possible to give a result showing that single experiments are (generically) sufficient for identification, in the analytic case. Some partial results can be obtained even if the precise nonlinearities are not known.<<ETX>>

[1]  Eduardo D. Sontag,et al.  Identi ability of discrete-time neural networks , 1993 .

[2]  Eduardo D. Sontag,et al.  UNIQUENESS OF WEIGHTS FOR NEURAL NETWORKS , 1993 .

[3]  Hava T. Siegelmann,et al.  Some recent results on computing with 'neural nets' , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[4]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[5]  Hava T. Siegelmann,et al.  On the computational power of neural nets , 1992, COLT '92.

[6]  Hava T. Siegelmann,et al.  Some results on computing with `neural nets , 1992 .

[7]  Eduardo Sontag,et al.  Algebraic theory of sign-linear systems , 1991, 1991 American Control Conference.

[8]  Eduardo Sontag,et al.  Turing computability with neural nets , 1991 .

[9]  Arjan van der Schaft,et al.  Non-linear dynamical control systems , 1990 .

[10]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[11]  James L. McClelland,et al.  Finite State Automata and Simple Recurrent Networks , 1989, Neural Computation.

[12]  Morris W. Hirsch,et al.  Convergent activation dynamics in continuous time networks , 1989, Neural Networks.

[13]  Robert Hecht-Nielsen,et al.  Theory of the backpropagation neural network , 1989, International 1989 Joint Conference on Neural Networks.

[14]  J. Farrell,et al.  Qualitative analysis of neural networks , 1988, 1988., IEEE International Symposium on Circuits and Systems.

[15]  Anthony J. Robinson,et al.  Static and Dynamic Error Propagation Networks with Application to Speech Coding , 1987, NIPS.

[16]  A. Isidori Nonlinear Control Systems: An Introduction , 1986 .

[17]  A. Isidori Nonlinear Control Systems , 1985 .

[18]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Stephen Grossberg,et al.  Absolute stability of global pattern formation and parallel memory storage by competitive neural networks , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  Eduardo Sontag On the Observability of Polynomial Systems, I: Finite-Time Problems , 1979 .

[21]  Eduardo Sontag On the Observability of Polynomial Systems. , 1977 .