Simplified neural networks for solving linear least squares and total least squares problems in real time

In this paper a new class of simplified low-cost analog artificial neural networks with on chip adaptive learning algorithms are proposed for solving linear systems of algebraic equations in real time. The proposed learning algorithms for linear least squares (LS), total least squares (TLS) and data least squares (DLS) problems can be considered as modifications and extensions of well known algorithms: the row-action projection-Kaczmarz algorithm and/or the LMS (Adaline) Widrow-Hoff algorithms. The algorithms can be applied to any problem which can be formulated as a linear regression problem. The correctness and high performance of the proposed neural networks are illustrated by extensive computer simulation results.

[1]  Jerry M. Mendel,et al.  Parallel Structured Networks for Solving a Wide Variety of Matrix Algebra Problems , 1992, J. Parallel Distributed Comput..

[2]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[3]  Malur K. Sundareshan,et al.  Exponential stability and a systematic synthesis of a neural network for quadratic minimization , 1991, Neural Networks.

[4]  Mauro Forti,et al.  Neural network for adaptive FIR filtering , 1990 .

[5]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[6]  Andrzej Cichocki,et al.  MOS Switched-Capacitor and Continuous-Time Integrated Circuits and Systems: Analysis and Design , 1989 .

[7]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[8]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[9]  C.R.K. Marrian,et al.  A neural net approach to discrete Hartley and Fourier transforms , 1989 .

[10]  A. Cichocki,et al.  Neural networks for solving systems of linear equations and related problems , 1992 .

[11]  M. Hanke,et al.  On the acceleration of Kaczmarz's method for inconsistent linear systems , 1990 .

[12]  Stuart Haber,et al.  A VLSI-efficient technique for generating multiple uncorrelated noise sources and its application to stochastic neural networks , 1991 .

[13]  Leon O. Chua,et al.  Neural networks for nonlinear programming , 1988 .

[14]  Andrzej Cichocki,et al.  Neural networks for linear inverse problems with incomplete data especially in applications to signal and image reconstruction , 1995, Neurocomputing.

[15]  Eric M. Dowling,et al.  The Data Least Squares Problem and Channel Equalization , 1993, IEEE Trans. Signal Process..

[16]  Sabine Van Huffel.  On the Significance of Nongeneric Total Least Squares Problems , 1992, SIAM J. Matrix Anal. Appl..

[17]  Sabine Van Huffel,et al.  Algebraic connections between the least squares and total least squares problems , 1989 .

[18]  K. S. Arun,et al.  A Unitarily Constrained Total Least Squares Problem in Signal Processing , 1992, SIAM J. Matrix Anal. Appl..

[19]  D. O’Leary.  Robust regression computation computation using iteratively reweighted least squares , 1990 .

[20]  John J. Hopfield,et al.  Simple 'neural' optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit , 1986 .

[21]  Willard L. Miranker,et al.  New techniques for fast hybrid solutions of systems of equations , 1989 .

[22]  R. Unbehauen,et al.  Neural networks for solving systems of linear equations. II. Minimax and least absolute value problems , 1992 .

[23]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[24]  G. Yin,et al.  Averaging procedures in adaptive filtering: an efficient approach , 1992 .

[25]  Musheng Wei,et al.  Algebraic relations between the total least squares and least squares problems with more than one solution , 1992 .