Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer

In many machine learning applications, there are multiple decision-makers involved, both automated and human. The interaction between these agents often goes unaddressed in algorithmic development. In this work, we explore a simple version of this interaction with a two-stage framework containing an automated model and an external decision-maker. The model can choose to say PASS, and pass the decision downstream, as explored in rejection learning. We extend this concept by proposing "learning to defer", which generalizes rejection learning by considering the effect of other agents in the decision-making process. We propose a learning algorithm which accounts for potential biases held by external decision-makers in a system. Experiments demonstrate that learning to defer can make systems not only more accurate but also less biased. Even when working with inconsistent or biased users, we show that deferring models still greatly improve the accuracy and/or fairness of the entire system.

[1]  Katrina Ligett,et al.  Learning Fair Classifiers: A Regularization-Inspired Approach , 2017, ArXiv.

[2]  Panagiotis G. Ipeirotis,et al.  Beat the Machine: Challenging Workers to Find the Unknown Unknowns , 2011, Human Computation.

[3]  Krishna P. Gummadi,et al.  Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment , 2016, WWW.

[4]  Toniann Pitassi,et al.  Fairness through awareness , 2011, ITCS '12.

[5]  Yee Whye Teh,et al.  The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables , 2016, ICLR.

[6]  Jun Sakuma,et al.  Fairness-Aware Classifier with Prejudice Remover Regularizer , 2012, ECML/PKDD.

[7]  Mehryar Mohri,et al.  Learning with Rejection , 2016, ALT.

[8]  Kush R. Varshney,et al.  On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products , 2016, Big Data.

[9]  Jenna Burrell,et al.  How the machine ‘thinks’: Understanding opacity in machine learning algorithms , 2016 .

[10]  Toniann Pitassi,et al.  Learning Fair Representations , 2013, ICML.

[11]  Timothy C. Y. Chan,et al.  Improving fairness in match play golf through enhanced handicap allocation , 2018, Journal of Sports Analytics.

[12]  C. K. Chow,et al.  An optimum character recognition system using decision functions , 1957, IRE Trans. Electron. Comput..

[13]  Jon M. Kleinberg,et al.  Inherent Trade-Offs in the Fair Determination of Risk Scores , 2016, ITCS.

[14]  Thomas Villmann,et al.  A Probabilistic Classifier Model with Adaptive Rejection Option Report 01 / 2016 , 2016 .

[15]  Aaron Roth,et al.  Fairness in Learning: Classic and Contextual Bandits , 2016, NIPS.

[16]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[17]  J. Roemer,et al.  Equality of Opportunity , 2013 .

[18]  Anca D. Dragan,et al.  Cooperative Inverse Reinforcement Learning , 2016, NIPS.

[19]  Aditya Krishna Menon,et al.  The cost of fairness in binary classification , 2018, FAT.

[20]  Lowell W. Busenitz,et al.  Differences between entrepreneurs and managers in large organizations: Biases and heuristics in strategic decision-making , 1997 .

[21]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[22]  S. Danziger,et al.  Extraneous factors in judicial decisions , 2011, Proceedings of the National Academy of Sciences.

[23]  Toon Calders,et al.  Classifying without discriminating , 2009, 2009 2nd International Conference on Computer, Control and Communication.

[24]  Xin Wang,et al.  IDK Cascades: Fast Deep Learning by Learning not to Overthink , 2017, UAI.

[25]  Julien Cornebise,et al.  Weight Uncertainty in Neural Network , 2015, ICML.

[26]  Kilian Q. Weinberger,et al.  On Calibration of Modern Neural Networks , 2017, ICML.

[27]  Sebastian Thrun,et al.  Dermatologist-level classification of skin cancer with deep neural networks , 2017, Nature.

[28]  Katrina Ligett,et al.  Penalizing Unfairness in Binary Classification , 2017 .

[29]  Alexandra Chouldechova,et al.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments , 2016, Big Data.

[30]  R. Dawes,et al.  Heuristics and Biases: Clinical versus Actuarial Judgment , 2002 .

[31]  Nathan Srebro,et al.  Equality of Opportunity in Supervised Learning , 2016, NIPS.

[32]  Krishna P. Gummadi,et al.  On Fairness, Diversity and Randomness in Algorithmic Decision Making , 2017, ArXiv.

[33]  Jon M. Kleinberg,et al.  On Fairness and Calibration , 2017, NIPS.

[34]  Danah Boyd,et al.  Fairness and Abstraction in Sociotechnical Systems , 2019, FAT.

[35]  Kelly Hannah-Moffat.  Actuarial Sentencing: An “Unsettled” Proposition , 2013 .

[36]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.