Image Feature Extraction and Denoising by Sparse Coding

Abstract: We show how sparse coding can be used to extract wavelet-like features from natural image data. Sparse coding is a method for finding a representation of image windows in which each of the components of the representation is only rarely significantly active. Such a representation is closely related to the techniques of independent component analysis and blind source separation. As an application of the sparse coding scheme, we show how to apply a soft-thresholding operator on the components of sparse coded noisy image windows in order to reduce Gaussian noise. The results outperform both Wiener and median filtering. Compared to wavelet transforms, methods based on sparse coding have the important benefit that the features are fully adapted to the training images and determined solely by their statistical properties, while the wavelet transformation relies heavily on certain abstract mathematical properties that may be only weakly related to the properties of the natural data.

[1]  A. Hyvarinen A family of fixed-point algorithms for independent component analysis , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[2]  I. Johnstone,et al.  Wavelet Shrinkage: Asymptopia? , 1995 .

[3]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[4]  Paul Wintz,et al.  Digital image processing (2nd ed.) , 1987 .

[5]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Horace Barlow,et al.  What is the computational goal of the neocortex , 1994 .

[7]  Aapo Hyvärinen,et al.  Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood , 1998, Neurocomputing.

[8]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[9]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[10]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[11]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[12]  Erkki Oja,et al.  Sparse Code Shrinkage: Denoising by Nonlinear Maximum Likelihood Estimation , 1998, NIPS.

[13]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[14]  Erkki Oja,et al.  Image feature extraction by sparse coding and independent component analysis , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[15]  Aapo Hyvärinen,et al.  Denoising of Sensory Data by Maximum Likelihood Estimation of Sparse Components , 1998 .

[16]  Edward H. Adelson,et al.  Noise removal via Bayesian wavelet coring , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[17]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[18]  Aapo Hyvärinen,et al.  Sparse Code Shrinkage: Denoising of Nongaussian Data by Maximum Likelihood Estimation , 1999, Neural Computation.

[19]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .