Robust Inference and Local Algorithms

We introduce a new feature to inference and learning which we call robustness. By robustness we intuitively model the case that the observation of the learner might be corrupted. We survey a new and novel approach to model such possible corruption as a zero-sum game between an adversary that selects the corruption and a leaner that predict the correct label. The corruption of the observations is done in a worse-case setting, by an adversary, where the main restriction is that the adversary is limited to use one of a fixed know class of modification functions. The main focus in this line of research is on efficient algorithms both for the inference setting and for the learning setting. In order to be efficient in the dimension of the domain, one cannot hope to inspect all the possible inputs. For this, we have to invoke local computation algorithms, that inspect only a logarithmic fraction of the domain per query.

[1]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[2]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[3]  Nathan Linial,et al.  Collective coin flipping, robust voting schemes and minima of Banzhaf values , 1985, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[4]  Nathan Linial,et al.  The influence of variables on Boolean functions , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[5]  Ming Li,et al.  Learning in the Presence of Malicious Errors , 1993, SIAM J. Comput..

[6]  J. Pearl Causality: Models, Reasoning and Inference , 2000 .

[7]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[8]  D. Angluin,et al.  Learning From Noisy Examples , 1988, Machine Learning.

[9]  Elchanan Mossel,et al.  Noise stability of functions with low influences: Invariance and optimality , 2005, IEEE Annual Symposium on Foundations of Computer Science.

[10]  Ronitt Rubinfeld,et al.  Fast Local Computation Algorithms , 2011, ICS.

[11]  Noga Alon,et al.  Space-efficient local computation algorithms , 2011, SODA.

[12]  Yishay Mansour,et al.  Converting Online Algorithms to Local Computation Algorithms , 2012, ICALP.

[13]  Yishay Mansour,et al.  A Local Computation Approximation Scheme to Maximum Matching , 2013, APPROX-RANDOM.

[14]  Dana Ron,et al.  Best of Two Local Models: Local Centralized and Local Distributed Algorithms , 2014, ArXiv.

[15]  Uriel Feige,et al.  Learning and inference in the presence of corrupted inputs , 2015, COLT.

[16]  Moshe Tennenholtz,et al.  Robust Probabilistic Inference , 2015, SODA.