Gradient adaptation under unit-norm constraints

We study gradient-based adaptive algorithms within parameter spaces specified by /spl par/w/spl par/=1, where /spl par//spl middot//spl par/ is any vector norm. Several approximate algorithms for this task have already been developed when /spl par/w/spl par/ is the L/sub 2/ norm. We derive general algorithm forms for arbitrary vector norms and relate them to true gradient procedures via their geometric structures. We also give algorithms that mitigate an inherent numerical instability for tangent-vector L/sub 2/-norm methods. Simulations showing the performance of the techniques for minor component analysis are provided.

[1]  Norman L. Owsley,et al.  Adaptive data orthogonalization , 1978, ICASSP.

[2]  T. Kailath,et al.  Least squares type algorithm for adaptive implementation of Pisarenko's harmonic retrieval method , 1982 .

[3]  Bede Liu,et al.  An iterative algorithm for locating the minimal eigenvector of a symmetric matrix , 1984, ICASSP.

[4]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[5]  Phillip A. Regalia.  An unbiased equation error identifier and reduced-order approximations , 1994, IEEE Trans. Signal Process..

[6]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[7]  K. C. Ho,et al.  Bias removal in equation-error adaptive IIR filters , 1995, IEEE Trans. Signal Process..

[8]  S. Douglas,et al.  On bias removal and unit norm constraints in equation error adaptive IIR filters , 1996, Conference Record of The Thirtieth Asilomar Conference on Signals, Systems and Computers.

[9]  Juha Karhunen,et al.  A Unified Neural Bigradient Algorithm for robust PCA and MCA , 1996, Int. J. Neural Syst..

[10]  Sun-Yuan Kung,et al.  Extraction of independent components from hybrid mixture: KuicNet learning algorithm and applications , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[11]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[12]  Shun-ichi Amari,et al.  Why natural gradient? , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[13]  S. Amari,et al.  A self-stabilized minor subspace rule , 1998, IEEE Signal Processing Letters.

[14]  Victor Solo,et al.  Performance analysis of adaptive eigenanalysis algorithms , 1998, IEEE Trans. Signal Process..