Low Mutual and Average Coherence Dictionary Learning Using Convex Approximation

In dictionary learning, a desirable property for the dictionary is to be of low mutual and average coherences. Mutual coherence is defined as the maximum absolute correlation between distinct atoms of the dictionary, whereas the average coherence is a measure of the average correlations. In this paper, we consider a dictionary learning problem regularized with the average coherence and constrained by an upper-bound on the mutual coherence of the dictionary. Our main contribution is then to propose an algorithm for solving the resulting problem based on convexly approximating the cost function over the dictionary. Experimental results demonstrate that the proposed approach has higher convergence rate and lower representation error (with a fixed sparsity parameter) than other methods, while yielding similar mutual and average coherence values.

[1]  Christian Jutten,et al.  Dictionary Learning for Sparse Representation: A Novel Approach , 2013, IEEE Signal Processing Letters.

[2]  Zhihui Zhu,et al.  A new framework for designing incoherent sparsifying dictionaries , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Joachim M. Buhmann,et al.  Learning Dictionaries With Bounded Self-Coherence , 2012, IEEE Signal Processing Letters.

[4]  J. Tropp On the conditioning of random subdictionaries , 2008 .

[5]  Stephen J. Wright,et al.  Computational Methods for Sparse Solution of Linear Inverse Problems , 2010, Proceedings of the IEEE.

[6]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Christian Jutten,et al.  Regularized low-coherence overcomplete dictionary learning for sparse signal decomposition , 2016, 2016 24th European Signal Processing Conference (EUSIPCO).

[8]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[9]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[10]  Mark D. Plumbley,et al.  Learning Incoherent Dictionaries for Sparse Approximation Using Iterative Projections and Rotations , 2013, IEEE Transactions on Signal Processing.

[11]  Miguel R. D. Rodrigues,et al.  Projection Design for Statistical Compressive Sensing: A Tight Frame Based Approach , 2013, IEEE Transactions on Signal Processing.

[12]  Christian Jutten,et al.  A New Algorithm for Dictionary Learning Based on Convex Approximation , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[13]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[14]  Christian Jutten,et al.  Joint Low Mutual and Average Coherence Dictionary Learning , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[15]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[16]  Jean Ponce,et al.  Task-Driven Dictionary Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Christian Jutten,et al.  Learning Overcomplete Dictionaries Based on Atom-by-Atom Updating , 2014, IEEE Transactions on Signal Processing.

[18]  Michael Elad,et al.  Dictionaries for Sparse Representation Modeling , 2010, Proceedings of the IEEE.

[19]  Thomas Strohmer,et al.  GRASSMANNIAN FRAMES WITH APPLICATIONS TO CODING AND COMMUNICATION , 2003, math/0301135.

[20]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[21]  Christian Jutten,et al.  A new algorithm for learning overcomplete dictionaries , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).