On the Intractability of Loading Neural Networks

Neural networks have been proposed as a tool for machine learning. In this role, a network is trained to recognize complex associations between inputs and outputs that were presented during a supervised training cycle. These associations are incorporated into the weights of the network, which encode a distributed representation of the information that was contained in the patterns. Once trained, the network will compute an input/output mapping which, if the training data was representative enough, will closely match the unknown rule which produced the original data. Massive parallelism of computation, as well as noise and fault tolerance, are often offered as justifications for the use of neural nets as learning paradigms.

[1]  Eduardo D. Sontag,et al.  Finiteness results for sigmoidal “neural” networks , 1993, STOC.

[2]  Xin Yao,et al.  Finding Approximate Solutions to NP-Hard Problems by Neural Networks is Hard , 1992, Inf. Process. Lett..

[3]  Joseph W. Goodman,et al.  On the power of neural networks for solving hard problems , 1990, J. Complex..

[4]  John T. Gill,et al.  Computational complexity of probabilistic Turing machines , 1974, STOC '74.

[5]  A. Barron Approximation and Estimation Bounds for Artificial Neural Networks , 1991, COLT '91.

[6]  Georg Schnitger,et al.  On the computational power of sigmoid versus Boolean threshold circuits , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[7]  Nimrod Megiddo,et al.  On the complexity of polyhedral separability , 1988, Discret. Comput. Geom..

[8]  Paul W. Goldberg,et al.  Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers , 1993, COLT '93.

[9]  Eduardo D. Sontag,et al.  Feedforward Nets for Interpolation and Classification , 1992, J. Comput. Syst. Sci..

[10]  Hava T. Siegelmann,et al.  Neural Networks With Real Weights: Analog Computational Complexity , 1992 .

[11]  Eduardo D. Sontag,et al.  Rate of approximation results motivated by robust neural network learning , 1993, COLT '93.

[12]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[13]  Georg Schnitger,et al.  The Power of Approximation: A Comparison of Activation Functions , 1992, NIPS.

[14]  Wolfgang Maass,et al.  Bounds for the computational power and learning complexity of analog neural nets , 1993, SIAM J. Comput..

[15]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[16]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[17]  Mihalis Yannakakis,et al.  On the complexity of local search , 1990, STOC '90.

[18]  Saburo Muroga,et al.  Threshold logic and its applications , 1971 .

[19]  Thomas Kailath,et al.  Classification of linearly nonseparable patterns by linear threshold elements , 1995, IEEE Trans. Neural Networks.

[20]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[21]  J. Stephen Judd,et al.  Neural network design and the complexity of learning , 1990, Neural network modeling and connectionism.

[22]  Xiao-Dong Zhang,et al.  Complexity Of Neural Network Learning In The Real Number Model , 1992, Workshop on Physics and Computation.

[23]  Jirí Benes,et al.  On neural networks , 1990, Kybernetika.

[24]  Hans Ulrich Simon,et al.  On learning ring-sum-expansions , 1990, COLT '90.

[25]  J. Stephen Judd,et al.  On the complexity of loading shallow neural networks , 1988, J. Complex..

[26]  L. Jones A Simple Lemma on Greedy Approximation in Hilbert Space and Convergence Rates for Projection Pursuit Regression and Neural Network Training , 1992 .

[27]  D SontagEduardo Feedforward nets for interpolation and classification , 1992 .

[28]  Hava T. Siegelmann,et al.  On the complexity of training neural networks with continuous activation functions , 1995, IEEE Trans. Neural Networks.

[29]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[30]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .