We describe a method for learning an overcomplete set of basis functions for the purpose of modeling sparse structure in images. The sparsity of the basis function coefficients is modeled with a mixture-of-Gaussians distribution. One Gaussian captures nonactive coefficients with a small-variance distribution centered at zero, while one or more other Gaussians capture active coefficients with a large-variance distribution. We show that when the prior is in such a form, there exist efficient methods for learning the basis functions as well as the parameters of the prior. The performance of the algorithm is demonstrated on a number of test cases and also on natural images. The basis functions learned on natural images are similar to those obtained with other methods, but the sparse form of the coefficient distribution is much better described. Also, since the parameters of the prior are adapted to the data, no assumption about sparse structure in the images need be made a priori, rather it is learned from the data.
[1]
Edward H. Adelson,et al.
Shiftable multiscale transforms
,
1992,
IEEE Trans. Inf. Theory.
[2]
David J. Field,et al.
Sparse coding with an overcomplete basis set: A strategy employed by V1?
,
1997,
Vision Research.
[3]
Terrence J. Sejnowski,et al.
The “independent components” of natural scenes are edge filters
,
1997,
Vision Research.
[4]
Axthonv G. Oettinger,et al.
IEEE Transactions on Information Theory
,
1998
.
[5]
J. H. Hateren,et al.
Independent component filters of natural images compared with simple cells in primary visual cortex
,
1998
.
[6]
Hagai Attias,et al.
Independent Factor Analysis
,
1999,
Neural Computation.
[7]
Bruno A. Olshausen,et al.
PROBABILISTIC FRAMEWORK FOR THE ADAPTATION AND COMPARISON OF IMAGE CODES
,
1999
.