Local privacy and statistical minimax rates

We study the tradeoff between privacy guarantees and the utility of statistical estimators under local differential privacy, where data remains private even from the statistician. We prove bounds on information-theoretic quantities that influence estimation rates as a function of the amount of privacy preserved. Our bounds can be viewed as quantitative data-processing inequalities that, combined with minimax techniques such as Le Cam's and Fano's methods, allow for precise characterization of statistical rates under local privacy constraints.

[1]  A. Wald.  Contributions to the Theory of Statistical Estimation and Testing Hypotheses , 1939 .

[2]  S L Warner,et al.  Randomized response: a survey technique for eliminating evasive answer bias. , 1965, Journal of the American Statistical Association.

[3]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[4]  George T. Duncan,et al.  Disclosure-Limited Data Dissemination , 1986 .

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[7]  Bin Yu.  Assouad, Fano, and Le Cam , 1997 .

[8]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[9]  Alexandre V. Evfimievski,et al.  Limiting privacy breaches in privacy preserving data mining , 2003, PODS.

[10]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[11]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[12]  Cynthia Dwork,et al.  Privacy, accuracy, and consistency too: a holistic solution to contingency table release , 2007, PODS.

[13]  L. Wasserman,et al.  A Statistical Framework for Differential Privacy , 2008, 0811.2501.

[14]  Sofya Raskhodnikova,et al.  What Can We Learn Privately? , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[15]  Eran Omri,et al.  Distributed Private Data Analysis: On Simultaneously Solving How and What , 2008, CRYPTO.

[16]  Martin J. Wainwright,et al.  Information-theoretic lower bounds on the oracle complexity of convex optimization , 2009, NIPS.

[17]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[18]  Cynthia Dwork,et al.  Differential privacy and robust statistics , 2009, STOC '09.

[19]  Kunal Talwar,et al.  On the geometry of differential privacy , 2009, STOC '10.

[20]  Toniann Pitassi,et al.  The Limits of Two-Party Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[21]  Guy N. Rothblum,et al.  A Multiplicative Weights Mechanism for Privacy-Preserving Data Analysis , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[22]  Stephen E. Fienberg,et al.  Differential Privacy and the Risk-Utility Tradeoff for Multi-dimensional Contingency Tables , 2010, Privacy in Statistical Databases.

[23]  Guy N. Rothblum,et al.  Boosting and Differential Privacy , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[24]  Adam D. Smith,et al.  Privacy-preserving statistical estimation with optimal convergence rates , 2011, STOC '11.

[25]  Anindya De,et al.  Lower Bounds in Differential Privacy , 2011, TCC.

[26]  Kamalika Chaudhuri,et al.  Convergence Rates for Differentially Private Statistical Estimation , 2012, ICML.

[27]  Michael I. Jordan,et al.  Local Privacy and Statistical Minimax Rates , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[28]  Larry A. Wasserman,et al.  Random Differential Privacy , 2011, J. Priv. Confidentiality.

[29]  Martin J. Wainwright,et al.  Privacy Aware Learning , 2012, JACM.