Unsupervised Rank Aggregation with Domain-Specific Expertise

Alexandre Klementiev, Dan Roth, Kevin Small, Ivan Titov

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Consider the setting where a panel of judges is repeatedly asked to (partially) rank sets of objects according to given criteria, and assume that the judges’ expertise depends on the objects’ domain. Learning to aggregate their rankings with the goal of producing a better joint ranking is a fundamental problem in many areas of Information Retrieval and Natural Language Processing, amongst others. However, supervised ranking data is generally difficult to obtain, especially if coming from multiple domains. Therefore, we propose a framework for learning to aggregate votes of constituent rankers with domain specific expertise without supervision. We apply the learning framework to the settings of aggregating full rankings and aggregating top-k lists, demonstrating significant improvements over a domain-agnostic baseline in both cases.
Original languageEnglish
Title of host publicationIJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009
PublisherAssociation for Computational Linguistics
Number of pages6
Publication statusPublished - 2009

Fingerprint Dive into the research topics of 'Unsupervised Rank Aggregation with Domain-Specific Expertise'. Together they form a unique fingerprint.

Cite this