A Dictionary Learning Method for Sparse Representation Using a Homotopy Approach

In this paper, we address the problem of dictionary learning for sparse representation. Considering the regularized form of the dictionary learning problem, we propose a method based on a homotopy approach, in which the regularization parameter is overall decreased along iterations. We estimate the value of the regularization parameter adaptively at each iteration based on the current value of the dictionary and the sparse coefficients, such that it preserves both sparse coefficients and dictionary optimality conditions. This value is, then, gradually decreased for the next iteration to follow a homotopy method. The results show that our method has faster implementation compared to recent dictionary learning methods, while overall it outperforms the other methods in recovering the dictionaries.

[1]  Joseph F. Murray,et al.  Dictionary Learning Algorithms for Sparse Representation , 2003, Neural Computation.

[2]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[3]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[4]  Michael Elad,et al.  Improving Dictionary Learning: Multiple Dictionary Updates and Coefficient Reuse , 2013, IEEE Signal Processing Letters.

[5]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[6]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[7]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[8]  Shijun Liao,et al.  Homotopy Analysis Method in Nonlinear Differential Equations , 2012 .

[9]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[10]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[11]  Mike E. Davies,et al.  Dictionary Learning for Sparse Approximations With the Majorization Method , 2009, IEEE Transactions on Signal Processing.

[12]  Mark D. Plumbley,et al.  Fast Dictionary Learning for Sparse Representations of Speech Signals , 2011, IEEE Journal of Selected Topics in Signal Processing.

[13]  Javier Portilla,et al.  Non-convex sparse optimization through deterministic annealing and applications , 2008, 2008 15th IEEE International Conference on Image Processing.

[14]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.