Optimizing Interventions via Offline Policy Evaluation: Studies in Citizen Science

Avi Segal, Yakov Gal, Ece Kamar, Eric Horvitz, Grant Miller

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Volunteers who help with online crowdsourcing such as citizen science tasks typically make only a few contributions before exiting. We propose a computational approach for increasing users’ engagement in such settings that is based on optimizing policies for displaying motivational messages to users. The approach, which we refer to as Trajectory Corrected Intervention (TCI), reasons about the tradeoff between the long-term influence of engagement messages on participants’ contributions and the potential risk of disrupting their current work. We combine model-based reinforcement learning with off-line policy evaluation to generate intervention policies, without relying on a fixed representation of the domain. TCI works iteratively to learn the best representation from a set of random intervention trials and to generate candidate intervention policies. It is able to refine selected policies off-line by exploiting the fact that users can only be interrupted once per session. We implemented TCI in the wild with Galaxy Zoo, one of the largest citizen science platforms on the web. We found that TCI was able to outperform the state-of-the-art intervention policy for this domain, and significantly increased the contributions of thousands of users. This work demonstrates the benefit of combining traditional AI planning with off-line policy methods to generate intelligent intervention strategies.
Original languageEnglish
Title of host publicationThirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)
Place of PublicationNew Orleans, Louisiana, USA
PublisherAAAI Press
Number of pages9
ISBN (Print)978-1-57735-800-8
Publication statusPublished - 7 Feb 2018
EventThirty-Second AAAI Conference on Artificial Intelligence - Hilton New Orleans Riverside, New Orleans, United States
Duration: 2 Feb 20187 Feb 2018

Publication series

PublisherAAAI Press
ISSN (Electronic)2374-3468


ConferenceThirty-Second AAAI Conference on Artificial Intelligence
Abbreviated titleAAAI 2018
Country/TerritoryUnited States
CityNew Orleans
Internet address


Dive into the research topics of 'Optimizing Interventions via Offline Policy Evaluation: Studies in Citizen Science'. Together they form a unique fingerprint.

Cite this