Highly overcomplete sparse coding

This paper explores sparse coding of natural images in the highly overcomplete regime. We show that as the overcompleteness ratio approaches l0x, new types of dictionary elements emerge beyond the classical Gabor function shape obtained from complete or only modestly overcomplete sparse coding. These more diverse dic tionaries allow images to be approximated with lower L1 norm (for a fixed SNR), and the coefficients exhibit steeper decay. We also evaluate the learned dictionaries in a denoising task, showing that higher degrees of overcompleteness yield modest gains in peformance.

[1]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[2]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[3]  H. Barlow.  Critical limiting factors in the design of the eye and visual cortex , 1981 .

[4]  Richard G. Baraniuk,et al.  Sparse Coding via Thresholding and Local Competition in Neural Circuits , 2008, Neural Computation.

[5]  Avideh Zakhor,et al.  Dictionary design for matching pursuit and application to motion-compensated video coding , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  D. Ruderman.  The statistics of natural images , 1994 .

[7]  B. Krauskopf,et al.  Proc of SPIE , 2003 .

[8]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[9]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[10]  David J. Field,et al.  Learning efficient linear codes for natural images: the roles of sparseness, overcompleteness, and statistical independence , 1996, Electronic Imaging.

[11]  Bruno A. Olshausen,et al.  Learning real and complex overcomplete representations from the statistics of natural images , 2009, Optical Engineering + Applications.

[12]  E. Candès,et al.  Ridgelets: a key to higher-dimensional intermittency? , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[13]  J. H. Hateren,et al.  Independent component filters of natural images compared with simple cells in primary visual cortex , 1998 .

[14]  Michael S. Lewicki,et al.  Robust Coding Over Noisy Overcomplete Channels , 2007, IEEE Transactions on Image Processing.

[15]  Pascal Frossard,et al.  Flexible motion-adaptive video coding with redundant expansions , 2006, IEEE Transactions on Circuits and Systems for Video Technology.