Interaction Design of a Semi-automatic Video Face Annotation System

In this paper, we propose an intelligent system that allows people to annotate face tracks of video content. To reduce the workload, we adopt two strategies in the system: 1) visually similar face tracks could be grouped together to reduce human reaction time; 2) face models could be learned to automatically recognize celebrity identities, leaving annotator only simple judgement tasks. With more precise face models, the recognized results require much less time for confirmation. Altogether, these strategies significantly reduce the workload of human annotation/confirmation. Experiments on a very large video repository prove the efficiency and effectiveness of the proposed system.

[1]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[2]  Yuxiao Hu,et al.  Efficient propagation for face annotation in family albums , 2004, MULTIMEDIA '04.

[3]  Yuan Li,et al.  High-Performance Rotation Invariant Multiview Face Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  D. Defays,et al.  An Efficient Algorithm for a Complete Link Method , 1977, Comput. J..

[6]  Dimitrios Androutsos,et al.  Logo and trademark detection in images using Color Wavelet Co-occurrence Histograms , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Yuandong Tian,et al.  EasyAlbum: an interactive photo annotation system based on face clustering and re-ranking , 2007, CHI.

[8]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Michael L. Creech,et al.  FotoFile: a consumer multimedia organization and retrieval system , 1999, CHI '99.

[11]  Mingjing Li,et al.  Automated annotation of human faces in family albums , 2003, MULTIMEDIA '03.

[12]  Marc Davis,et al.  Photo annotation on a camera phone , 2004, CHI EA '04.

[13]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[14]  Thomas S. Huang,et al.  ImageGrouper: Search, Annotate and Organize Images by Groups , 2002, VISUAL.

[15]  Yuandong Tian,et al.  A Face Annotation Framework with Partial Clustering and Interactive Labeling , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Suh-Yin Lee,et al.  Recent Advances in Visual Information Systems , 2002, Lecture Notes in Computer Science.

[17]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Tohru Hoshi,et al.  Communication and Collaboration Support Systems , 2005 .

[19]  Peter Gejgus,et al.  Face tracking in color video sequences , 2003, SCCG '03.

[20]  Thomas S. Huang,et al.  Relevance feedback: a power tool for interactive content-based image retrieval , 1998, IEEE Trans. Circuits Syst. Video Technol..

[21]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[22]  Benjamin B. Bederson,et al.  Semi-automatic photo annotation strategies using event based clustering and clothing based person recognition , 2007, Interact. Comput..

[23]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[24]  Borko Furht,et al.  Content-Based Image and Video Retrieval , 2002, Multimedia Systems and Applications Series.

[25]  Jonathan Grudin.  Multimedia Annotation: An Unsuccessful Tool Becomes a Successful Framework , 2005 .