Neural networks for solving systems of linear equations. II. Minimax and least absolute value problems

For pt.I see ibid., vol.39, no.2, p.124-38 (1992). The minimax (L/sub infinity /- or Chebyshev norm) and the least absolute value (L/sub 1/-norm) optimization criteria for linear parameter estimation problems are reformulated as constrained minimization problems. For these problems appropriate energy (Lyapunov) functions are constructed which enable the problems to be mapped into systems of nonlinear ordinary differential equations. On the basis of these systems of equations, analog neuronlike network architectures are proposed and their properties are discussed. The proposed circuit structures exhibit a high degree of modularity, and in most cases a relatively small number of basic building blocks (processing units) are required to implement effective and powerful optimization algorithms. The validity and performance of the architectures are illustrated by extensive computer simulations and CMOS implementations of a general-purpose network architecture are considered. >

[1]  W. Steiger,et al.  Least Absolute Deviations: Theory, Applications and Algorithms , 1984 .

[2]  Paulo J. G. Lisboa,et al.  Special Issue on Neural Networks , 1993 .

[3]  S. Osowski.  Neural networks in minimax programming , 1990, IEEE International Workshop on Cellular Neural Networks and their Applications.

[4]  N. Otsu,et al.  Restoration of images with missing high-frequency components by minimizing the L1 norm of the solution vector. , 1985, Applied optics.

[5]  S. Treitel,et al.  Fast l p solution of large, sparse, linear systems: application to seismic travel time tomography , 1988 .

[6]  D. O’Leary.  Robust regression computation computation using iteratively reweighted least squares , 1990 .

[7]  Philip E. Gill,et al.  Practical optimization , 1981 .

[8]  A. Conn,et al.  Primal methods are better than dual methods for solving overdetermined linear systems in the l ∞ sense? , 1989 .

[9]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Adriaan van den Bos,et al.  Nonlinear least-absolute-values and minimax model fitting , 1985, Autom..

[11]  Andrzej Cichocki,et al.  Switched-capacitor artificial neural networks for nonlinear optimization with constraints , 1990, IEEE International Symposium on Circuits and Systems.

[12]  C.R.K. Marrian,et al.  A neural net approach to discrete Hartley and Fourier transforms , 1989 .

[13]  Andrzej Cichocki,et al.  Switched-capacitor neural networks for differential optimization , 1991, Int. J. Circuit Theory Appl..

[14]  J. Bee Bednar,et al.  Fast algorithms for lpdeconvolution , 1985, IEEE Trans. Acoust. Speech Signal Process..

[15]  A. Dax.  The Minimax Solution of Linear Equations Subject to Linear Constraints , 1989 .

[16]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[17]  R. Perfetti,et al.  Neural network for real-time synthesis of FIR filters , 1989 .

[18]  Rabab K. Ward.  An on-line adaptation for discrete l_{1} linear estimation , 1984 .

[19]  Edgar Sanchez-Sinencio,et al.  Nonlinear switched capacitor 'neural' networks for optimization problems , 1990 .

[20]  John W. Bandler,et al.  A nonlinear l_1 optimization algorithm for design, modeling, and diagnosis of networks , 1987 .

[21]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[22]  N. Abdelmalek.  Chebyshev and L1 solutions of overdetermined systems of linear equations with bounded variables , 1986 .

[23]  I. Barrodale,et al.  High-Resolution Signal and Noise Field Estimation Using the L 1 (Least Absolute Values) Norm , 1987 .

[24]  Yannis Tsividis,et al.  A reconfigurable VLSI neural network , 1992 .

[25]  Werner A. Stahel,et al.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[26]  Xiaobo Sharon Hu,et al.  Maximal solution of linear systems of equations and an application in VLSI , 1990, IEEE International Symposium on Circuits and Systems.

[27]  Jerry M. Mendel,et al.  Structured trainable networks for matrix algebra , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[28]  Gene H. Golub,et al.  Matrix computations , 1983 .

[29]  H J Caulfield,et al.  Superconvergence of hybrid optoelectronic processors. , 1987, Applied optics.

[30]  J. Vandewalle,et al.  Analysis and properties of the generalized total least squares problem AX≈B when some or all columns in A are subject to error , 1989 .

[31]  Mohammed Ismail,et al.  Analog VLSI Implementation of Neural Systems , 2011, The Kluwer International Series in Engineering and Computer Science.

[32]  Elwood T. Olsen,et al.  L1 and L∞ minimization via a variant of Karmarkar's algorithm , 1989, IEEE Trans. Acoust. Speech Signal Process..

[33]  Bing J. Sheu,et al.  Analog floating-gate synapses for general-purpose VLSI neural computation , 1991 .

[34]  J. Ramirez-Angulo,et al.  Architectures for analog VLSI implementation of neural networks for solving linear equations with inequality constraints , 1992, [Proceedings] 1992 IEEE International Symposium on Circuits and Systems.

[35]  Dimitris Anastassiou,et al.  Switched-capacitor neural networks , 1987 .

[36]  T. Ulrych,et al.  A linear programming approach to the estimation of the power spectra of harmonic processes , 1982 .

[37]  Michael Peter Kennedy,et al.  Unifying the Tank and Hopfield linear programming circuit and the canonical nonlinear programming circuit of Chua and Lin , 1987 .

[38]  A. Cichocki,et al.  Neural networks for solving systems of linear equations and related problems , 1992 .

[39]  Mauro Forti.  A neural network for signal decomposition problems , 1991, Int. J. Circuit Theory Appl..

[40]  Willard L. Miranker,et al.  New techniques for fast hybrid solutions of systems of equations , 1989 .

[41]  B. Linares-Barranco,et al.  VLSI implementation of a transconductance mode continuous BAM with on chip learning and dynamic analog memory , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[42]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[43]  Andrzej Cichocki,et al.  MOS Switched-Capacitor and Continuous-Time Integrated Circuits and Systems: Analysis and Design , 1989 .

[44]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .