Image annotation using multi-label correlated Green's function

Image annotation has been an active research topic in the recent years due to its potentially large impact on both image understanding and web/database image search. In this paper, we target at solving the automatic image annotation problem in a novel semi-supervised learning framework. A novel multi-label correlated Green's function approach is proposed to annotate images over a graph. The correlations among labels are integrated into the objective function which improves the performance significantly. We also propose a new adaptive decision boundary method for multi-label assignment to deal with the difficulty of label assignment in most of the existing rank-based multi-label classification algorithms. Instead of setting the threshold heuristically or by experience, our method principally compute it upon the prior knowledge in the training data. We perform our methods on three commonly used image annotation testing data sets. Experimental results show significant improvements on classification performance over four other state-of-the-art methods. As a general semi-supervised learning framework, other local feature based image annotation methods could be easily incorporated into our framework to improve the performance.

[1]  Tommi S. Jaakkola,et al.  Partially labeled classification with Markov random walks , 2001, NIPS.

[2]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[3]  Zoubin Ghahramani,et al.  Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions , 2003, ICML.

[4]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[5]  Edward Y. Chang,et al.  CBSA: content-based soft annotation for multimodal image retrieval using Bayes point machines , 2003, IEEE Trans. Circuits Syst. Video Technol..

[6]  Yoram Singer,et al.  Log-Linear Models for Label Ranking , 2003, NIPS.

[7]  Yoram Singer,et al.  BoosTexter: A Boosting-based System for Text Categorization , 2000, Machine Learning.

[8]  Volker Tresp,et al.  Multi-label informed latent semantic indexing , 2005, SIGIR '05.

[9]  Yihong Gong,et al.  Multi-labelled classification using maximum entropy method , 2005, SIGIR '05.

[10]  Gustavo Carneiro,et al.  Formulating semantic image annotation as a supervised learning problem , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[12]  Chris H. Q. Ding,et al.  A learning framework using Green's function and kernel regularization with application to recommender system , 2007, KDD '07.

[13]  Rong Jin,et al.  Semi-supervised SVM batch mode active learning for image retrieval , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Tao Mei,et al.  Joint multi-label multi-instance learning for image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Tao Mei,et al.  Graph-based semi-supervised learning with multi-label , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[16]  Gang Chen,et al.  Semi-supervised Multi-label Learning by Solving a Sylvester Equation , 2008, SDM.

[17]  Horst Bischof,et al.  Semi-supervised boosting using visual similarity learning , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Yong Wang,et al.  Coherent image annotation by learning semantic distance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.