CRF framework for supervised preference aggregation

We develop a flexible Conditional Random Field framework for supervised preference aggregation, which combines preferences from multiple experts over items to form a distribution over rankings. The distribution is based on an energy comprised of unary and pairwise potentials allowing us to effectively capture correlations between both items and experts. We describe procedures for learning in this modelnand demonstrate that inference can be done much more efficiently thannin analogous models. Experiments on benchmark tasks demonstrate significant performance gains over existing rank aggregation methods.

[1]  Rushed Kanawati,et al.  Supervised rank aggregation approach for link prediction in complex networks , 2012, WWW.

[2]  Hugo Larochelle,et al.  Learning to rank by aggregating expert preferences , 2012, CIKM.

[3]  William W. Cohen,et al.  Semi-Markov Conditional Random Fields for Information Extraction , 2004, NIPS.

[4]  John D. Lafferty,et al.  Cranking: Combining Rankings Using Conditional Probability Models on Permutations , 2002, ICML.

[5]  Tao Qin,et al.  LETOR: Benchmark Dataset for Research on Learning to Rank for Information Retrieval , 2007 .

[6]  Ronald Fagin,et al.  Efficient similarity search and classification via rank aggregation , 2003, SIGMOD '03.

[7]  Prem Melville,et al.  Supervised Rank Aggregation for Predicting Influencers in Twitter , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[8]  Christopher J. C. Burges,et al.  From RankNet to LambdaRank to LambdaMART: An Overview , 2010 .

[9]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS THE METHOD OF PAIRED COMPARISONS , 1952 .

[10]  Tao Qin,et al.  Global Ranking Using Continuous Conditional Random Fields , 2008, NIPS.

[11]  Yasubumi Sakakibara,et al.  RNA secondary structural alignment with conditional random fields , 2005, ECCB/JBI.

[12]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[13]  Alexander J. Smola,et al.  Learning Graph Matching , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  R. Plackett The Analysis of Permutations , 1975 .

[15]  Jaime G. Carbonell,et al.  Protein Fold Recognition Using Segmentation Conditional Random Fields (SCRFs) , 2006, J. Comput. Biol..

[16]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[17]  Dan Roth,et al.  Integer linear programming inference for conditional random fields , 2005, ICML.

[18]  Javed A. Aslam,et al.  Condorcet fusion for improved retrieval , 2002, CIKM '02.

[19]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS , 1952 .

[20]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[21]  P. Moran On the method of paired comparisons. , 1947, Biometrika.

[22]  Noah A. Smith,et al.  Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions , 2010, NAACL.

[23]  Tao Qin,et al.  Supervised rank aggregation , 2007, WWW '07.

[24]  David A. McAllester,et al.  Generalization bounds and consistency for latent-structural probit and ramp loss , 2011, MLSLP.

[25]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[26]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[27]  Tao Qin,et al.  A New Probabilistic Model for Rank Aggregation , 2010, NIPS.

[28]  R. Luce,et al.  Individual Choice Behavior: A Theoretical Analysis. , 1960 .

[29]  Dan Roth,et al.  Unsupervised rank aggregation with distance-based models , 2008, ICML '08.

[30]  Fei Wang,et al.  Semi-supervised ranking aggregation , 2008, CIKM '08.

[31]  Jeff A. Bilmes,et al.  Consensus ranking under the exponential model , 2007, UAI.

[32]  Craig Boutilier,et al.  Learning Mallows Models with Pairwise Preferences , 2011, ICML.

[33]  Maksims Volkovs,et al.  BoltzRank: learning to maximize expected ranking gain , 2009, ICML '09.

[34]  Charles L. A. Clarke,et al.  Reciprocal rank fusion outperforms condorcet and individual rank learning methods , 2009, SIGIR.

[35]  David Mease,et al.  A Penalized Maximum Likelihood Approach for the Ranking of College Football Teams Independent of Victory Margins , 2003 .

[36]  Javed A. Aslam,et al.  Models for metasearch , 2001, SIGIR '01.

[37]  C. L. Mallows NON-NULL RANKING MODELS. I , 1957 .

[38]  David F. Gleich,et al.  Rank aggregation via nuclear norm minimization , 2011, KDD.

[39]  John Guiver,et al.  Bayesian inference for Plackett-Luce ranking models , 2009, ICML '09.