Edinburgh Research Explorer

Mechanical cheat

Research output: Contribution to journalArticlepeer-review

Original languageEnglish
Pages (from-to)26
JournalNew Scientist
Volume211
Issue number2822
DOIs
Publication statusPublished - 22 Jul 2011

Abstract

Jim Giles and Zena Lovino's article "Scams, mischief and fraud in the crowd" (9th July, p20-21) describes some abuses of Amazon's Mechanical Turk by those offering online jobs. Abuse, however, also frequently occurs in the opposite direction. Researchers paying turkers to fill in multi-choice questionnaires find that a large proportion of responders game the system. In order to maximise their income, these gamers answer the questionnaire as quickly as possible by clicking the most convenient button. In order to exclude these misleading responses from their experimental data, researchers have to employ a variety of devices, such as planting a few questions with obvious answers then ignoring all answers from anyone that gets these planted questions wrong. Researchers who have not taken such precautions will find that referees reject their experimental results.

ID: 165330432