Double sparsity for multi-frame super resolution

A number of image super resolution algorithms based on the sparse coding have successfully implemented multi-frame super resolution in recent years. In order to utilize multiple low-resolution observations, both accurate image registration and sparse coding are required. Previous study on multi-frame super resolution based on sparse coding firstly apply block matching for image registration, followed by sparse coding to enhance the image resolution. In this paper, these two problems are solved by optimizing a single objective function. The proposed formulation not only has a mathematically interesting structure, called the double sparsity, but also yields comparable or improved numerical performance to conventional methods.

[1]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[2]  Hideitsu Hino,et al.  Multi-frame image super resolution based on sparse coding , 2015, Neural Networks.

[3]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[4]  Shie Mannor,et al.  The Sample Complexity of Dictionary Learning , 2010, COLT.

[5]  Xuelong Li,et al.  A multi-frame image super-resolution method , 2010, Signal Process..

[6]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[7]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[8]  Fei Gao,et al.  Deep Multimodal Distance Metric Learning Using Click Constraints for Image Ranking , 2017, IEEE Transactions on Cybernetics.

[9]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[11]  Wen Gao,et al.  Robust Estimation of 3D Human Poses from a Single Image , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[13]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[14]  Zhengtao Yu,et al.  Multiple graph regularized sparse coding and multiple hypergraph regularized sparse coding for image representation , 2015, Neurocomputing.

[15]  Peng Wang,et al.  Super Resolution Reconstruction via Multiple Frames Joint Learning , 2011, 2011 International Conference on Multimedia and Signal Processing.

[16]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[17]  Neil D. B. Bruce,et al.  Sparse coding in early visual representation: From specific properties to general principles , 2016, Neurocomputing.

[18]  Hideitsu Hino,et al.  Double Sparse Multi-Frame Image Super Resolution , 2015, ArXiv.

[19]  Haitao Yu,et al.  Graph Regularized Sparsity Discriminant Analysis for face recognition , 2016, Neurocomputing.

[20]  Meng Wang,et al.  Multimodal Deep Autoencoder for Human Pose Recovery , 2015, IEEE Transactions on Image Processing.

[21]  Lei Zhang,et al.  Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization , 2010, IEEE Transactions on Image Processing.

[22]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[23]  Rong Zhang,et al.  Remote sensing image compression based on double-sparsity dictionary learning and universal trellis coded quantization , 2013, 2013 IEEE International Conference on Image Processing.

[24]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[25]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[26]  Jun Yu,et al.  Click Prediction for Web Image Reranking Using Multimodal Sparse Coding , 2014, IEEE Transactions on Image Processing.

[27]  Michael Elad,et al.  On Single Image Scale-Up Using Sparse-Representations , 2010, Curves and Surfaces.

[28]  Michael Elad,et al.  Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation , 2010, IEEE Transactions on Signal Processing.

[29]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[30]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[31]  Meng Wang,et al.  Image-Based Three-Dimensional Human Pose Recovery by Multiview Locality-Sensitive Sparse Retrieval , 2015, IEEE Transactions on Industrial Electronics.

[32]  Thomas S. Huang,et al.  Image super-resolution as sparse representation of raw image patches , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  David J. Field,et al.  How Close Are We to Understanding V1? , 2005, Neural Computation.

[34]  Alexander G. Gray,et al.  Sparsity-Based Generalization Bounds for Predictive Sparse Coding , 2013, ICML.

[35]  Jun Zhou,et al.  Local and global regularized sparse coding for data representation , 2016, Neurocomputing.

[36]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[37]  Hideitsu Hino,et al.  Doubly sparse structure in image super resolution , 2016, 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP).

[38]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[39]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[40]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[41]  Masatoshi Okutomi,et al.  Multi-Parameter Simultaneous Estimation on Area-Based Matching , 2006, International Journal of Computer Vision.

[42]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.