Interactive Modeling of Concept Drift and Errors in Relevance Feedback

Antti Kangasrääsiö, Yi Chen, Dorota Glowacka, Samuel Kaski

Research output: Chapter in Book/Report/Conference proceedingConference contribution


In exploratory search tasks, users usually start with considerable uncertainty about their search goals, and so the search intent of the user may be volatile as the user is constantly learning and reformulating her search hypothesis during the search. This may lead to a noticeable concept drift in the relevance feedback given by the user. We formulate a Bayesian regression model for predicting the accuracy of each individual user feedback and thus find outliers in the feedback data set. To accompany this model, we introduce a timeline interface that visualizes the feedback history to the user and gives her suggestions on which past feedback is likely in need of adjustment. This interface also allows the user to adjust the feedback accuracy inferences made by the model. Simulation experiments demonstrate that the performance of the new user model outperforms a simpler baseline and that the performance approaches that of an oracle, given a small amount of additional user interaction. A user study shows that the proposed modeling technique, combined with the timeline interface, made it easier for the users to notice and correct mistakes in their feedback, resulted in better and more diverse recommendations, allowed users to easier find items they liked, and was more understandable.
Original languageEnglish
Title of host publicationProceedings of the 2016 Conference on User Modeling Adaptation and Personalization
Place of PublicationNew York, NY, USA
Number of pages9
ISBN (Print)978-1-4503-4368-8
Publication statusPublished - 13 Jul 2016
Event2016 Conference on User Modeling Adaptation and Personalization - Halifax, Canada
Duration: 13 Jul 201616 Jul 2016

Publication series

NameUMAP '16


Conference2016 Conference on User Modeling Adaptation and Personalization
Abbreviated titleUMAP 2016
Internet address


Dive into the research topics of 'Interactive Modeling of Concept Drift and Errors in Relevance Feedback'. Together they form a unique fingerprint.

Cite this