Separable Dictionary Learning for Convolutional Sparse Coding via Split Updates

Existing methods for constructing separable 2D dictionary filter banks approximate a set of $K$ non-separable filters via a linear combination of $R\ll K$ separable filters. This approach involves the inefficiency of learning an initial set of non-separable filters, and places an upper bound on the quality of the separable filter banks. In this paper, we propose a method to directly learn a set of $K$ separable dictionary filters from a given image training set by drawing ideas from convolutional dictionary learning (CDL) methods. We show that the separable filters obtained by our method match the performance of an equivalent number of non-separable filters. Furthermore, the computational performance of our learning method is shown to be substantially faster than a state-of-the-art non-separable CDL method for large numbers of filters or large training sets.

[1]  Kévin Degraux,et al.  Online convolutional dictionary learning for multimodal imaging , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[2]  André Uschmajew,et al.  Finding a low-rank basis in a matrix subspace , 2015, Math. Program..

[3]  Filip Sroubek,et al.  Fast convolutional sparse coding using matrix inversion lemma , 2016, Digit. Signal Process..

[4]  Wotao Yin,et al.  First- and Second-Order Methods for Online Convolutional Dictionary Learning , 2018, SIAM J. Imaging Sci..

[5]  L. Breiman Better subset regression using the nonnegative garrote , 1995 .

[6]  Jean Ponce,et al.  Sparse Modeling for Image and Vision Processing , 2014, Found. Trends Comput. Graph. Vis..

[7]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[8]  Paul Rodríguez Alternating optimization low-rank expansion algorithm to estimate a linear combination of separable filters to approximate 2D filter banks , 2016, 2016 50th Asilomar Conference on Signals, Systems and Computers.

[9]  Brendt Wohlberg,et al.  Convolutional Dictionary Learning , 2017, Computer Vision.

[10]  Brendt Wohlberg,et al.  Efficient Algorithms for Convolutional Sparse Representations , 2016, IEEE Transactions on Image Processing.

[11]  Vincent Lepetit,et al.  Learning Separable Filters , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Wotao Yin,et al.  Online convolutional dictionary learning , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[13]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[14]  Andrew Zisserman,et al.  Speeding up Convolutional Neural Networks with Low Rank Expansions , 2014, BMVC.

[15]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Brendt Wohlberg,et al.  Fast convolutional sparse coding with separable filters , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Morten Mørup,et al.  Transformation invariant sparse coding , 2011, 2011 IEEE International Workshop on Machine Learning for Signal Processing.

[18]  Brendt Wohlberg,et al.  Subproblem coupling in convolutional dictionary learning , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[19]  José Carlos Príncipe,et al.  A fast proximal method for convolutional sparse coding , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[20]  Quentin Barthelemy,et al.  Shift & 2D Rotation Invariant Sparse Coding for Multivariate Signals , 2012, IEEE Transactions on Signal Processing.

[21]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[22]  Graham W. Taylor,et al.  Adaptive deconvolutional networks for mid and high level feature learning , 2011, 2011 International Conference on Computer Vision.