A Harmonic Mean Linear Discriminant Analysis for Robust Image Classification

Linear Discriminant Analysis (LDA) is a widely-used supervised dimensionality reduction method in computer vision and pattern recognition. In null space based LDA (NLDA), a well-known LDA extension, between-class distance is maximized in the null space of the within-class scatter matrix. However, there are some limitations in NLDA. Firstly, for many data sets, null space of within-class scatter matrix does not exist, thus NLDA is not applicable to those datasets. Secondly, NLDA uses arithmetic mean of between-class distances and gives equal consideration to all between-class distances, which makes larger between-class distances can dominate the result and thus limits the performance of NLDA. In this paper, we propose a harmonic mean based Linear Discriminant Analysis, Multi-Class Discriminant Analysis (MCDA), for image classification, which minimizes the reciprocal of weighted harmonic mean of pairwise between-class distance. More importantly, MCDA gives higher priority to maximize small between-class distances. MCDA can be extended to multi-label dimension reduction. Results on 7 single-label data sets and 4 multi-label data sets show that MCDA has consistently better performance than 10 other single-label approaches and 4 other multi-label approaches in terms of classification accuracy, macro and micro average F1 score.

[1]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[2]  Jason Weston,et al.  A kernel method for multi-labelled classification , 2001, NIPS.

[3]  Jiebo Luo,et al.  Learning multi-label scene classification , 2004, Pattern Recognit..

[4]  ZhouZhi-Hua,et al.  Multilabel dimensionality reduction via dependence maximization , 2010 .

[5]  Hongdong Li,et al.  A Convex Programming Approach to the Trace Quotient Problem , 2007, ACCV.

[6]  Xiangliang Zhang,et al.  TideWatch: Fingerprinting the cyclicality of big data workloads , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[7]  David M. Rocke,et al.  Dimension Reduction for Classification with Gene Expression Microarray Data , 2006, Statistical applications in genetics and molecular biology.

[8]  Feiping Nie,et al.  A Closed Form Solution to Multi-View Low-Rank Regression , 2015, AAAI.

[9]  Jieping Ye,et al.  Extracting shared subspace for multi-label classification , 2008, KDD.

[10]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[11]  Trevor Hastie,et al.  Regularized linear discriminant analysis and its application in microarrays. , 2007, Biostatistics.

[12]  Jieping Ye,et al.  Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems , 2005, J. Mach. Learn. Res..

[13]  Feiping Nie,et al.  Orthogonal vs. uncorrelated least squares discriminant analysis for feature extraction , 2012, Pattern Recognit. Lett..

[14]  Chris H. Q. Ding,et al.  Accelerating Deep Learning with Shrinkage and Recall , 2016, 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS).

[15]  Liana L. Fong,et al.  Analysis and Modeling of Social Influence in High Performance Computing Workloads , 2011, Euro-Par.

[16]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[17]  YeJieping Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems , 2005 .

[18]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[19]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[20]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[21]  Xiaoou Tang,et al.  Dual-space linear discriminant analysis for face recognition , 2004, CVPR 2004.

[22]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Chris H. Q. Ding,et al.  A Semi-definite Positive Linear Discriminant Analysis and Its Applications , 2012, 2012 IEEE 12th International Conference on Data Mining.

[24]  Xiangliang Zhang,et al.  Virtual machine migration in an over-committed cloud , 2012, 2012 IEEE Network Operations and Management Symposium.

[25]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[26]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[27]  J. B. Rosen,et al.  Lower Dimensional Representation of Text Data Based on Centroids and Least Squares , 2003 .

[28]  Feiping Nie,et al.  Trace Ratio Criterion for Feature Selection , 2008, AAAI.

[29]  Zhi-Hua Zhou,et al.  Multilabel dimensionality reduction via dependence maximization , 2008, TKDD.

[30]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2006, IEEE Transactions on Neural Networks.

[31]  Jieping Ye,et al.  Discriminant Analysis for Dimensionality Reduction: An Overview of Recent Developments , 2010 .

[32]  Chris H. Q. Ding,et al.  Kernel Alignment Inspired Linear Discriminant Analysis , 2014, ECML/PKDD.

[33]  Chris H. Q. Ding,et al.  Multi-label Linear Discriminant Analysis , 2010, ECCV.

[34]  Volker Tresp,et al.  Multi-label informed latent semantic indexing , 2005, SIGIR '05.

[35]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[36]  Feiping Nie,et al.  Neighborhood MinMax Projections , 2007, IJCAI.

[37]  Feiping Nie,et al.  Semi-supervised orthogonal discriminant analysis via label propagation , 2009, Pattern Recognit..

[38]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.

[39]  R. Chellappa,et al.  Subspace Linear Discriminant Analysis for Face RecognitionW , 1999 .

[40]  Nebojsa Jojic,et al.  LOCUS: learning object classes with unsupervised segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.