A Framework for Optimizing Paper Matching

At the heart of many scientific conferences is the problem of matching submitted papers to suitable reviewers. Arriving at a good assignment is a major and important challenge for any conference organizer. In this paper we propose a framework to optimize paper-to-reviewer assignments. Our framework uses suitability scores to measure pairwise affinity between papers and reviewers. We show how learning can be used to infer suitability scores from a small set of provided scores, thereby reducing the burden on reviewers and organizers. We frame the assignment problem as an integer program and propose several variations for the paper-to-reviewer matching domain. We also explore how learning and matching interact. Experiments on two conference data sets examine the performance of several learning methods as well as the effectiveness of the matching formulations.

[1]  L. Shapley,et al.  College Admissions and the Stability of Marriage , 1962 .

[2]  R. Zeckhauser,et al.  The Efficient Allocation of Individuals to Positions , 1979, Journal of Political Economy.

[3]  A. Roth.  The Evolution of the Labor Market for Medical Interns and Residents: A Case Study in Game Theory , 1984, Journal of Political Economy.

[4]  Jakob Nielsen,et al.  Automating the assignment of submitted manuscripts to reviewers , 1992, SIGIR '92.

[5]  Douglas B. Terry,et al.  Using collaborative filtering to weave an information tapestry , 1992, CACM.

[6]  William W. Cohen,et al.  Recommending Papers by Mining the Web , 1999 .

[7]  Salem Benferhat,et al.  Conference paper assignment , 2001, Int. J. Intell. Syst..

[8]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[9]  W. Bruce Croft,et al.  LDA-based document models for ad-hoc retrieval , 2006, SIGIR.

[10]  Michael J. Pazzani,et al.  Mining for proposal reviewers: lessons learned at the national science foundation , 2006, KDD '06.

[11]  M. de Rijke,et al.  Formal models for expert finding in enterprise corpora , 2006, SIGIR.

[12]  Andrew McCallum,et al.  Expertise modeling for matching papers with reviewers , 2007, KDD '07.

[13]  Judy Goldsmith,et al.  The AI conference paper assignment problem , 2007, AAAI 2007.

[14]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.

[15]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[16]  Alexander J. Smola,et al.  COFI RANK - Maximum Margin Matrix Factorization for Collaborative Ranking , 2007, NIPS.

[17]  Johan Bollen,et al.  An algorithm to determine peer-reviewers , 2006, CIKM '08.

[18]  Ruslan Salakhutdinov,et al.  Bayesian probabilistic matrix factorization using Markov chain Monte Carlo , 2008, ICML '08.

[19]  Yoshua Bengio,et al.  Classification using discriminative restricted Boltzmann machines , 2008, ICML '08.

[20]  Camillo J. Taylor,et al.  On the Optimal Assignment of Conference Papers to Reviewers , 2008 .

[21]  Jin Yu,et al.  Exponential Family Graph Matching and Ranking , 2009, NIPS.

[22]  Kurt Mehlhorn,et al.  Assigning Papers to Referees , 2009, Algorithmica.

[23]  Neil D. Lawrence,et al.  Non-linear matrix factorization with Gaussian processes , 2009, ICML '09.

[24]  Geoffrey E. Hinton,et al.  Replicated Softmax: an Undirected Topic Model , 2009, NIPS.

[25]  Naren Ramakrishnan,et al.  Recommender systems for the conference paper assignment problem , 2009, RecSys '09.

[26]  Jie Tang,et al.  Expertise Matching via Constraint-Based Optimization , 2010, Web Intelligence.

[27]  Jean-Marc Petit,et al.  Web Intelligence and Intelligent Agent Technology , 2011 .

[28]  J. Gabszewicz,et al.  On Gale and Shapley "college admissions and the stability of marriage" , 2012 .

[29]  L. S. Shapley,et al.  College Admissions and the Stability of Marriage , 2013, Am. Math. Mon..