Images, Frames, and Connectionist Hierarchies

The representation of hierarchically structured knowledge in systems using distributed patterns of activity is an abiding concern for the connectionist solution of cognitively rich problems. Here, we use statistical unsupervised learning to consider semantic aspects of structured knowledge representation. We meld unsupervised learning notions formulated for multilinear models with tensor product ideas for representing rich information. We apply the model to images of faces.

[1]  Geoffrey E. Hinton,et al.  Generative models for discovering sparse distributed representations. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[2]  Y. Amit,et al.  An integrated network for invariant visual detection and recognition , 2003, Vision Research.

[3]  Joshua B. Tenenbaum,et al.  Separating Style and Content with Bilinear Models , 2000, Neural Computation.

[4]  R. Morris.  Parallel Distributed Processing: Implications for Psychology and Neurobiology , 1990 .

[5]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[6]  Peter Dayan,et al.  Neural Models for Part-Whole Hierarchies , 1996, NIPS.

[7]  D. Mackay.  The Epistemological Problem for Automata , 1956 .

[8]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[9]  Avi Pfeffer,et al.  Probabilistic Frame-Based Systems , 1998, AAAI/IAAI.

[10]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[11]  Tony A. Plate,et al.  Holographic Reduced Representation: Distributed Representation for Cognitive Structures , 2003 .

[12]  D. Long.  Probabilistic Models of the Brain. , 2002 .

[13]  Christoph von der Malsburg,et al.  Pattern recognition by labeled graph matching , 1988, Neural Networks.

[14]  Stuart J. Russell,et al.  BLOG: Relational Modeling with Unknown Objects , 2004 .

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[16]  Shimon Edelman,et al.  Representation and recognition in vision , 1999 .

[17]  C. P. Dolan.  Tensor manipulation networks: connectionist and symbolic approaches to comprehension, learning, and planning , 1989 .

[18]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[19]  Alessandro Sperduti,et al.  Labelling Recursive Auto-associative Memory , 1994, Connect. Sci..

[20]  D. Gentner,et al.  Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences , 1997, Cognitive Psychology.

[21]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[22]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[23]  Yali Amit,et al.  POP: Patchwork of Parts Models for Object Recognition , 2007, International Journal of Computer Vision.

[24]  Geoffrey E. Hinton,et al.  Using Generative Models for Handwritten Digit Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Andrea J. van Doorn,et al.  The Generic Bilinear Calibration-Estimation Problem , 2004, International Journal of Computer Vision.

[26]  Geoffrey E. Hinton.  Mapping Part-Whole Hierarchies into Connectionist Networks , 1990, Artif. Intell..

[27]  Tony A. Plate,et al.  Holographic reduced representations , 1995, IEEE Trans. Neural Networks.

[28]  Takeo Kanade,et al.  Object Detection Using the Statistics of Parts , 2004, International Journal of Computer Vision.

[29]  Bernt Schiele,et al.  Scale-Invariant Object Categorization Using a Scale-Adaptive Mean-Shift Search , 2004, DAGM-Symposium.

[30]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[31]  Alexandre Pouget,et al.  Basis Functions for Object-Centered Representations , 2003, Neuron.

[32]  L. Lathauwer,et al.  Dimensionality reduction in higher-order signal processing and rank-(R1,R2,…,RN) reduction in multilinear algebra , 2004 .

[33]  Bernt Schiele,et al.  Probabilistic object recognition using multidimensional receptive field histograms , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[34]  T. Poggio.  A theory of how the brain might work. , 1990, Cold Spring Harbor symposia on quantitative biology.

[35]  Rajesh P. N. Rao,et al.  Bilinear Sparse Coding for Invariant Vision , 2005, Neural Computation.

[36]  David Mumford,et al.  Neuronal Architectures for Pattern-theoretic Problems , 1995 .

[37]  Dmitri A. Rachkovskij,et al.  Binding and Normalization of Binary Sparse Distributed Representations by Context-Dependent Thinning , 2001, Neural Computation.

[38]  Bernt Schiele,et al.  Interleaving Object Categorization and Segmentation , 2006, Cognitive Vision Systems.

[39]  Pietro Perona,et al.  A Probabilistic Approach to Object Recognition Using Local Photometry and Global Geometry , 1998, ECCV.

[40]  Bernt Schiele,et al.  Recognition without Correspondence using Multidimensional Receptive Field Histograms , 2004, International Journal of Computer Vision.

[41]  Geoffrey E. Hinton.  Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems , 1991 .

[42]  Pietro Perona,et al.  A Bayesian approach to unsupervised one-shot learning of object categories , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[43]  Ross W. Gayler,et al.  Multiplicative Binding, Representation Operators & Analogy , 1998 .

[44]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[45]  Eric Mjolsness.  Bayesian Inference on Visual Grammars by Neural Nets that Optimize , 2004 .

[46]  Geoffrey E. Hinton,et al.  Learning Distributed Representations of Concepts Using Linear Relational Embedding , 2001, IEEE Trans. Knowl. Data Eng..

[47]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[48]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Geoffrey E. Hinton.  Learning distributed representations of concepts. , 1989 .

[50]  Antonio Torralba,et al.  Learning hierarchical models of scenes, objects, and parts , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[51]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[52]  M. Riesenhuber,et al.  Face processing in humans is compatible with a simple shape–based model of vision , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[53]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[54]  Daniel P. Huttenlocher,et al.  Spatial priors for part-based recognition using statistical models , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  Martin A. Fischler,et al.  The Representation and Matching of Pictorial Structures , 1973, IEEE Transactions on Computers.

[56]  Tomaso Poggio,et al.  Image Representations for Visual Learning , 1996, Science.

[57]  C. Burly,et al.  Face Localization via Shape Statistics , 1995 .

[58]  Franz J. Kurfess,et al.  Connectionist Symbol Processing , 1994 .

[59]  B. Schiele,et al.  Interleaved Object Categorization and Segmentation , 2003, BMVC.

[60]  Pentti Kanerva,et al.  Binary Spatter-Coding of Ordered K-Tuples , 1996, ICANN.

[61]  T. Sejnowski,et al.  Spatial Transformations in the Parietal Cortex Using Basis Functions , 1997, Journal of Cognitive Neuroscience.

[62]  L. Abbott,et al.  Invariant visual responses from attentional gain fields. , 1997, Journal of neurophysiology.

[63]  D. V. van Essen,et al.  Responses in area V4 depend on the spatial relationship between stimulus and attention. , 1996, Journal of neurophysiology.

[64]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[65]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[66]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[67]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[68]  Demetri Terzopoulos,et al.  Multilinear subspace analysis of image ensembles , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[69]  Demetri Terzopoulos,et al.  Multilinear independent components analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[70]  Bruce Bridgeman,et al.  A theory of visual stability across saccadic eye movements , 1994, Behavioral and Brain Sciences.