A multilayer neural network with piecewise-linear structure and back-propagation learning

A multilayer neural network which is given a two-layer piecewise-linear structure for every cascaded section is proposed. The neural networks have nonlinear elements that are neither sigmoidal nor of a signum type. Each nonlinear element is an absolute value operator. It is almost everywhere differentiable, which makes back-propagation feasible in a digital setting. Both the feedforward signal propagation and the backward coefficient update rules belong to the class of regular iterative algorithms. This form of neural network specializes in functional approximation and is anticipated to have applications in control, communications, and pattern recognition.

[1]  Leon O. Chua,et al.  Canonical piecewise-linear representation , 1988 .

[2]  Philip D. Wasserman,et al.  Neural computing - theory and practice , 1989 .

[3]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[4]  Jorma Laaksonen,et al.  Variants of self-organizing maps , 1990, International 1989 Joint Conference on Neural Networks.

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  B. Widrow,et al.  The truck backer-upper: an example of self-learning in neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[7]  Derrick H. Nguyen,et al.  Neural networks for self-learning control systems , 1990 .

[8]  Leon O. Chua,et al.  Canonical piecewise-linear modeling , 1986 .

[9]  Jin Luo,et al.  Computing motion using analog and binary resistive networks , 1988, Computer.

[10]  J.-N. Lin,et al.  Adaptive nonlinear digital filter with canonical piecewise-linear structure , 1990 .

[11]  Bernard Widrow,et al.  Layered neural nets for pattern recognition , 1988, IEEE Trans. Acoust. Speech Signal Process..

[12]  Carver Mead,et al.  Analog VLSI and neural systems , 1989 .

[13]  L. Chua,et al.  A generalized canonical piecewise-linear representation , 1990 .

[14]  Sailesh K. Rao,et al.  What is a Systolic Algorithm? , 1986, Photonics West - Lasers and Applications in Science and Engineering.

[15]  Yoh-Han Pao,et al.  Adaptive pattern recognition and neural networks , 1989 .

[16]  B. Widrow,et al.  Stationary and nonstationary learning characteristics of the LMS adaptive filter , 1976, Proceedings of the IEEE.

[17]  Donald F. Specht,et al.  Probabilistic neural networks and the polynomial Adaline as complementary techniques for classification , 1990, IEEE Trans. Neural Networks.

[18]  Thomas Kailath,et al.  Model-free distributed learning , 1990, IEEE Trans. Neural Networks.

[19]  Jeffrey M. Camhi,et al.  THE ESCAPE BEHAVIOR OF THE COCKROACH PERIPLANETA AMERICANA. II. DETECTION OF NATURAL PREDATORS BY AIR DISPLACEMENT , 1978 .