Learning Horizontal Connections in a Sparse Coding Model of Natural Images

It has been shown that adapting a dictionary of basis functions to the statistics of natural images so as to maximize sparsity in the coefficients results in a set of dictionary elements whose spatial properties resemble those of V1 (primary visual cortex) receptive fields. However, the resulting sparse coefficients still exhibit pronounced statistical dependencies, thus violating the independence assumption of the sparse coding model. Here, we propose a model that attempts to capture the dependencies among the basis function coefficients by including a pairwise coupling term in the prior over the coefficient activity states. When adapted to the statistics of natural images, the coupling terms learn a combination of facilitatory and inhibitory interactions among neighboring basis functions. These learned interactions may offer an explanation for the function of horizontal connections in V1 in terms of a prior over natural images.

[1]  A. Hyvärinen,et al.  A multi-layer sparse coding network learns contour coding from natural images , 2002, Vision Research.

[2]  Geoffrey E. Hinton,et al.  A Learning Algorithm for Boltzmann Machines , 1985, Cogn. Sci..

[3]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[4]  Ohad Ben-Shahar,et al.  Geometrical Computations Explain Projection Patterns of Long-Range Horizontal Connections in Visual Cortex , 2004, Neural Computation.

[5]  Geoffrey E. Hinton,et al.  Learning Causally Linked Markov Random Fields , 2005, AISTATS.

[6]  Michael J. Berry,et al.  Weak pairwise correlations imply strongly correlated network states in a neural population , 2005, Nature.

[7]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[8]  Zhou Wang,et al.  Structural Approaches to Image Quality Assessment , 2005 .

[9]  Terrence J. Sejnowski,et al.  Soft Mixer Assignment in a Hierarchical Generative Model of Natural Scene Statistics , 2006, Neural Computation.

[10]  D. Fitzpatrick,et al.  Orientation Selectivity and the Arrangement of Horizontal Connections in Tree Shrew Striate Cortex , 1997, The Journal of Neuroscience.

[11]  Eero P. Simoncelli,et al.  Statistical Modeling of Images with Fields of Gaussian Scale Mixtures , 2006, NIPS.

[12]  K. Jarrod Millman,et al.  Learning Sparse Codes with a Mixture-of-Gaussians Prior , 1999, NIPS.

[13]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[14]  Rajat Raina,et al.  Self-taught learning: transfer learning from unlabeled data , 2007, ICML '07.

[15]  Richard G. Baraniuk,et al.  Neurally plausible sparse coding via competitive algorithms , 2007 .

[16]  Eero P. Simoncelli,et al.  Random Cascades on Wavelet Trees and Their Use in Analyzing and Modeling Natural Images , 2001 .

[17]  Li Zhaoping,et al.  Border Ownership from Intracortical Interactions in Visual Area V2 , 2005, Neuron.

[18]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[19]  A. Grinvald,et al.  Relationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. , 1993, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[21]  Michael S. Lewicki,et al.  A Hierarchical Bayesian Model for Learning Nonlinear Statistical Regularities in Nonstationary Natural Signals , 2005, Neural Computation.

[22]  Aapo Hyvärinen,et al.  Statistical models of images and early vision , 2005 .

[23]  D. Fitzpatrick The functional organization of local circuits in visual cortex: insights from the study of tree shrew striate cortex. , 1996, Cerebral cortex.

[24]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[25]  Gerhard Krieger,et al.  The atoms of vision: Cartesian or polar? , 1999 .