Robust Logistic Regression and Classification

We consider logistic regression with arbitrary outliers in the covariate matrix. We propose a new robust logistic regression algorithm, called RoLR, that estimates the parameter through a simple linear programming procedure. We prove that RoLR is robust to a constant fraction of adversarial outliers. To the best of our knowledge, this is the first result on estimating logistic regression model when the covariate matrix is corrupted with any performance guarantees. Besides regression, we apply RoLR to solving binary classification problems where a fraction of training samples are corrupted.

[1]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[2]  F. Hampel.  The Influence Curve and Its Role in Robust Estimation , 1974 .

[3]  D. Pregibon.  Logistic Regression Diagnostics , 1981 .

[4]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[5]  D. Pregibon.  Resistant fits for some commonly used logistic models with medical application. , 1982, Biometrics.

[6]  W. Johnson.  Influence measures for logistic regression: Another point of view , 1985 .

[7]  D. Ruppert,et al.  Optimally bounded score functions for generalized linear models with applications to logistic regression , 1986 .

[8]  J. Copas.  Binary Regression Models for Contaminated Data , 1988 .

[9]  R. Carroll,et al.  Conditionally Unbiased Bounded-Influence Estimation in General Regression Models, with Applications to Generalized Linear Models , 1989 .

[10]  Ana M. Bianco,et al.  Robust Estimation in the Logistic Regression Model , 1996 .

[11]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[12]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[13]  Honglak Lee,et al.  Efficient L1 Regularized Logistic Regression , 2006, AAAI.

[14]  Po-Ling Loh,et al.  High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity , 2011, NIPS.

[15]  Yaniv Plan,et al.  Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach , 2012, IEEE Transactions on Information Theory.

[16]  Shie Mannor,et al.  Robust Sparse Regression under Adversarial Corruption , 2013, ICML.

[17]  Christopher D. Manning,et al.  Robust Logistic Regression using Shift Parameters (Long Version) , 2013 .

[18]  Christopher D. Manning,et al.  Robust Logistic Regression using Shift Parameters , 2014, ACL.