Convolutional Sparse Coding with Overlapping Group Norms

The most widely used form of convolutional sparse coding uses an $\ell_1$ regularization term. While this approach has been successful in a variety of applications, a limitation of the $\ell_1$ penalty is that it is homogeneous across the spatial and filter index dimensions of the sparse representation array, so that sparsity cannot be separately controlled across these dimensions. The present paper considers the consequences of replacing the $\ell_1$ penalty with a mixed group norm, motivated by recent theoretical results for convolutional sparse representations. Algorithms are developed for solving the resulting problems, which are quite challenging, and the impact on the performance of the denoising problem is evaluated. The mixed group norms are found to perform very poorly in this application. While their performance is greatly improved by introducing a weighting strategy, such a strategy also improves the performance obtained from the much simpler and computationally cheaper $\ell_1$ norm.

[1]  Brendt Wohlberg,et al.  Efficient convolutional sparse coding , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  M. Kowalski.  Sparse regression using mixed norms , 2009 .

[3]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[4]  Vishal M. Patel,et al.  Convolutional Sparse Coding-based Image Decomposition , 2016, BMVC.

[5]  Brendt Wohlberg,et al.  Efficient Algorithms for Convolutional Sparse Representations , 2016, IEEE Transactions on Image Processing.

[6]  Ping-Keng Jao,et al.  Informed monaural source separation of music based on convolutional sparse coding , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Bruno Torrésani,et al.  Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients , 2009, Signal Image Video Process..

[8]  Brendt Wohlberg,et al.  Sparse Overcomplete Denoising: Aggregation Versus Global Optimization , 2017, IEEE Signal Processing Letters.

[9]  Brendt Wohlberg,et al.  Convolutional Sparse Representations with Gradient Penalties , 2017, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Julien Mairal,et al.  Optimization with Sparsity-Inducing Penalties , 2011, Found. Trends Mach. Learn..

[11]  Wotao Yin,et al.  Group sparse optimization by alternating direction method , 2013, Optics & Photonics - Optical Engineering + Applications.

[12]  Lei Zhang,et al.  Convolutional Sparse Coding for Image Super-Resolution , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[14]  Michael Elad,et al.  Working Locally Thinking Globally - Part II: Stability and Algorithms for Convolutional Sparse Coding , 2016, ArXiv.

[15]  Michael Elad,et al.  Working Locally Thinking Globally - Part I: Theoretical Guarantees for Convolutional Sparse Coding , 2016, ArXiv.

[16]  Brendt Wohlberg,et al.  Piano music transcription with fast convolutional sparse coding , 2015, 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP).

[17]  Brendt Wohlberg,et al.  Convolutional sparse representation of color images , 2016, 2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI).

[18]  Rong Jin,et al.  Exclusive Lasso for Multi-task Feature Selection , 2010, AISTATS.

[19]  Brendt Wohlberg,et al.  SPORCO: A Python package for standard and convolutional sparse representations , 2017, SciPy.

[20]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[21]  Vishal M. Patel,et al.  Convolutional Sparse and Low-Rank Coding-Based Rain Streak Removal , 2017, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[22]  Brendt Wohlberg,et al.  Convolutional Laplacian sparse coding , 2016, 2016 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI).

[23]  Won-Ki Jeong,et al.  Compressed sensing reconstruction of dynamic contrast enhanced MRI using GPU-accelerated convolutional sparse coding , 2016, 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI).

[24]  Yann LeCun,et al.  Structured sparse coding via lateral inhibition , 2011, NIPS.

[25]  Graham W. Taylor,et al.  Deconvolutional networks , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.

[27]  Brendt Wohlberg,et al.  Piano Transcription With Convolutional Sparse Lateral Inhibition , 2017, IEEE Signal Processing Letters.

[28]  Brendt Wohlberg.  Convolutional sparse representations as an image model for impulse noise restoration , 2016, 2016 IEEE 12th Image, Video, and Multidimensional Signal Processing Workshop (IVMSP).

[29]  Rabab Kreidieh Ward,et al.  Image Fusion With Convolutional Sparse Representation , 2016, IEEE Signal Processing Letters.

[30]  F. Bach,et al.  Optimization with Sparsity-Inducing Penalties (Foundations and Trends(R) in Machine Learning) , 2011 .

[31]  Brendt Wohlberg,et al.  ADMM Penalty Parameter Selection by Residual Balancing , 2017, ArXiv.

[32]  Michael Elad,et al.  Working Locally Thinking Globally: Theoretical Guarantees for Convolutional Sparse Coding , 2017, IEEE Transactions on Signal Processing.