Optimizing Classifier Performance Via the Wilcoxon-Mann-Whitney Statistic

Cross entropy and mean squared error are typical cost functions used to optimize classi er performance. The goal of the optimization is usually to achieve the best correct classi cation rate. However, for many two-class real-world problems, the ROC curve is a more meaningful performance measure. We demonstrate that minimizing cross entropy or mean squared error does not necessarily maximize the area under the ROC curve (AUC). We then consider alternative objective functions for training a classi er to maximize the AUC directly. We propose an objective function that is an approximation to the Wilcoxon-Mann-Whitney statistic, which is equivalent to AUC. The proposed objective function is di erentiable, so gradient-based methods can be used to train the classi er. After discussing the improved results of the new objective function over several UCI data sets, we apply the new objective function to real-world customer behavior prediction problems for a wireless service provider and a cable service provider, and achieve reliable and signi cant improvements in the ROC curve.