Image Retrieval with Textual Label Similarity Features

This article presents a knowledge-based solution for retrieving English descriptions of images. We analyse the errors made by a baseline system that relies on term frequency, and we find that the task requires deeper semantic representation. Our solution is to perform incremental, task-driven development of an ontology. Ontological features are then applied in a machine-learning algorithm for ranking candidate image descriptions. This work demonstrates the advantage of combining knowledge-based and statistical approaches for text retrieval, and it establishes the important result that an empirically tuned task-specific ontology performs better than a domain-general resource like WordNet, even on previously unseen examples. Copyright © 2015 John Wiley & Sons, Ltd.

[1]  Rada Mihalcea,et al.  Measuring the semantic relatedness between words and images , 2011, IWCS.

[2]  Rada Mihalcea,et al.  Text Mining for Automatic Image Tagging , 2010, COLING.

[3]  Michael Collins,et al.  Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms , 2002, EMNLP.

[4]  Cyrus Rashtchian,et al.  Collecting Image Annotations Using Amazon’s Mechanical Turk , 2010, Mturk@HLT-NAACL.

[5]  W. Bruce Croft,et al.  Evaluation of an inference network-based retrieval model , 1991, TOIS.

[6]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[7]  Thierry Pun,et al.  The Truth about Corel - Evaluation in Image Retrieval , 2002, CIVR.

[8]  Mark J. Huiskes,et al.  The MIR flickr retrieval evaluation , 2008, MIR '08.

[9]  Paul Clough,et al.  The IAPR TC-12 Benchmark: A New Evaluation Resource for Visual Information Systems , 2006 .

[10]  Cyrus Rashtchian,et al.  Every Picture Tells a Story: Generating Sentences from Images , 2010, ECCV.

[11]  Andrew Hickl,et al.  A Discourse Commitment-Based Framework for Recognizing Textual Entailment , 2007, ACL-PASCAL@ACL.

[12]  Jerry R. Hobbs,et al.  Elaborating a Knowledge Base for Deep Lexical Semantics , 2011, IWCS.

[13]  W. Bruce Croft,et al.  A Language Modeling Approach to Information Retrieval , 1998, SIGIR Forum.

[14]  Nicola Guarino,et al.  Sweetening WORDNET with DOLCE , 2003, AI Mag..

[15]  Nicola Guarino,et al.  The Won-derWeb Library of Foundational Ontologies , 2002 .

[16]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[17]  Nicola Guarino,et al.  The WonderWeb Library of Foundational Ontologies Preliminary Report , 2002 .

[18]  Scott E. Fahlman,et al.  Marker-Passing Inference in the Scone Knowledge-Base System , 2006, KSEM.

[19]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[20]  Ken Barker,et al.  Towards Context Aware Emotional Intelligence in Machines: Computing Contextual Appropriateness of Affective States , 2009, IJCAI.