Novel algorithms for learning overcomplete dictionaries

Using a Bayesian framework based on an assumption of a convex/Schur-convex (CSC) log-prior, together with an associated affine-scaling transformation (AST) optimization algorithm, a signal vector, y, can be succinctly represented within a very overcomplete m/spl times/n dictionary of representation vectors a/sub i/, A=[a/sub 1/,...,a/sub n/], n/spl Gt/m, dictionary by obtaining a sparse solution, x, to the linear inverse problem Ax/spl ap/y/spl dot/. In this paper we outline how novel approximate maximum likelihood (AML) and maximum a posteriori (MAP) over-complete dictionary learning algorithms can be developed within the CSC/AST framework.

[1]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[2]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[3]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.

[4]  K. Kreutz-Delgado,et al.  A General Approach to Sparse Basis Selection : Majorization , Concavity , and Affine Scaling — — — — — – , 1997 .

[5]  Terrence J. Sejnowski,et al.  Learning Nonlinear Overcomplete Representations for Efficient Coding , 1997, NIPS.

[6]  K. Kreutz-Delgado,et al.  Deriving algorithms for computing sparse solutions to linear inverse problems , 1997, Conference Record of the Thirty-First Asilomar Conference on Signals, Systems and Computers (Cat. No.97CB36136).

[7]  Bhaskar D. Rao,et al.  Measures and algorithms for best basis selection , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[8]  K. Kreutz-Delgado,et al.  Basis selection in the presence of noise , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[9]  Bhaskar D. Rao,et al.  Signal processing with the sparseness constraint , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[10]  Bhaskar D. Rao,et al.  Sparse basis selection, ICA, and majorization: towards a unified perspective , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[11]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..