Semantic image clustering using object relation network

This paper presents a novel method to organize a collection of images into a hierarchy of clusters based on image semantics. Given a group of raw images with no metadata as input, our method describes the semantics of each image with a bag-of-semantics model (i.e., a set of meaningful descriptors), which is derived from the image's Object Relation Network [5] - an expressive graph model representing rich semantics for image objects and their relations. We adopt the class hierarchies in a guide ontology as different levels of lenses to view the bag-of-semantics models. Image clusters are automatically extracted by grouping images with the same bag-of-semantics viewed through a certain lens. With a series of coarse-to-fine lenses, images are clustered in a top-down hierarchical manner. In addition, given that users can have different perspectives regarding how images should be clustered, our method allows each user to control the clustering process while browsing, and thus dynamically adjusts the clustering result according to the user's preferences.

[1]  Shiri Gordon,et al.  Applying the information bottleneck principle to unsupervised clustering of discrete and continuous image representations , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[2]  Cor J. Veenman,et al.  Kernel Codebooks for Scene Categorization , 2008, ECCV.

[3]  Wei-Ying Ma,et al.  Locality preserving clustering for image database , 2004, MULTIMEDIA '04.

[4]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[5]  Marco Brambilla,et al.  A revenue sharing mechanism for federated search and advertising , 2012, WWW.

[6]  Yixin Chen,et al.  Image Categorization by Learning and Reasoning with Regions , 2004, J. Mach. Learn. Res..

[7]  Wei-Ying Ma,et al.  Hierarchical clustering of WWW image search results using visual, textual and link information , 2004, MULTIMEDIA '04.

[8]  Axel Pinz,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[9]  Ying Liu,et al.  Semantic Clustering for Region-Based Image Retrieval , 2007, Ninth IEEE International Symposium on Multimedia Workshops (ISMW 2007).

[10]  Wei-Ying Ma,et al.  IGroup: web image search results clustering , 2006, MM '06.

[11]  Jianguo Zhang,et al.  The PASCAL Visual Object Classes Challenge , 2006 .

[12]  Tao Qin,et al.  Web image clustering by consistent utilization of visual features and surrounding texts , 2005, MULTIMEDIA '05.

[13]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Viktor K. Prasanna,et al.  A bag-of-semantics model for image clustering , 2013, The Visual Computer.

[15]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Wei-Ying Ma,et al.  Iteratively clustering web images based on link and attribute reinforcements , 2005, ACM Multimedia.

[18]  Yixin Chen,et al.  CLUE: cluster-based retrieval of images by unsupervised learning , 2005, IEEE Transactions on Image Processing.

[19]  Yixin Chen,et al.  A sparse support vector machine approach to region-based image categorization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[20]  Kerry Rodden,et al.  Does organisation by similarity assist image browsing? , 2001, CHI.

[21]  Philippe A. Palanque,et al.  Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 2014, International Conference on Human Factors in Computing Systems.

[22]  Andrew J. Davison,et al.  Active Matching , 2008, ECCV.

[23]  Wei-Ying Ma,et al.  IGroup: a web image search engine with semantic clustering of search results , 2006, MM '06.

[24]  Viktor K. Prasanna,et al.  Understanding web images by object relation network , 2012, WWW.

[25]  Andrew Zisserman,et al.  Scene Classification Via pLSA , 2006, ECCV.