Don’t recommend the obvious: Estimate probability ratios

R Pellegrini, Wenjie Zhao, Iain Murray

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Sequential recommender systems are becoming widespread in the online retail and streaming industry. These systems are often trained to predict the next item given a sequence of a user’s recent actions, and standard evaluation metrics reward systems that can identify the most probable items that might appear next. However, some recent papers instead evaluate recommendation systems with popularity-sampled metrics, which measure how well the model can find a user’s next item when hidden amongst generally-popular items. We argue that these popularity-sampled metrics are more appropriate for recommender systems, because the most probable items for a user often include generally-popular items. If the probability that a customer will watch Toy Story is not much more probable than for the average customer, then the movie isn’t especially relevant for them and we should not recommend it. This paper shows that optimizing popularity-sampled metrics is closely related to estimating point-wise mutual information (PMI). We propose and compare two techniques to fit PMI directly, which both improve popularity-sampled metrics for state-of-the-art recommender systems. The improvements are large compar
Original languageEnglish
Title of host publicationProceedings of the 16th ACM Conference on Recommender Systems
EditorsJennifer Golbeck, F. Maxwell Harper, Vanessa Murdock, Michael Ekstrand, Bracha Shapira, Justin Basilico, Keld Lundgaard, Even Oldridge
PublisherACM Association for Computing Machinery
Number of pages10
ISBN (Electronic)978-1-4503-9278-5
Publication statusPublished - 13 Sep 2022
Event16th ACM Conference on Recommender Systems - Seattle, United States
Duration: 18 Sep 202223 Sep 2022


Conference16th ACM Conference on Recommender Systems
Abbreviated titleRecSys 2022
Country/TerritoryUnited States
Internet address


  • Algorithms
  • Evaluation metrics and Studies
  • Novel Machine Learning Approaches


Dive into the research topics of 'Don’t recommend the obvious: Estimate probability ratios'. Together they form a unique fingerprint.

Cite this