Recruiting Participants With Programming Skills: A Comparison of Four Crowdsourcing Platforms and a CS Student Mailing List

Mohammad Tahaei, Kami Vaniea

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Reliably recruiting participants with programming skills is an ongoing challenge for empirical studies involving software development technologies, often leading to the use of crowdsourcing platforms and computer science (CS) students. In this work, we use five existing survey instruments to explore the programming skills, privacy and security attitudes, and secure development self-efficacy of participants from a CS student mailing list and four crowdsourcing platforms (Appen, Clickworker, MTurk, and Prolific). We recruited 613 participants who claimed to have programming skills and assessed recruitment channels regarding costs, quality, programming skills, as well as privacy and security attitudes. We find that 27% of crowdsourcing participants, 40% of crowdsourcing participants who self-report to be developers, and 89% of CS students answered all programming skill questions correctly. CS students were the most cost-effective recruitment channel and rated themselves lower than crowdsourcing participants about secure development self-efficacy.

Original languageEnglish
Title of host publicationCHI 2022 - Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery, Inc
Number of pages15
ISBN (Electronic)9781450391573
DOIs
Publication statusPublished - 29 Apr 2022
Event2022 CHI Conference on Human Factors in Computing Systems, CHI 2022 - Virtual, Online, United States
Duration: 30 Apr 20225 May 2022

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2022 CHI Conference on Human Factors in Computing Systems, CHI 2022
Country/TerritoryUnited States
CityVirtual, Online
Period30/04/225/05/22

Keywords

  • crowdsourcing
  • datasets
  • developers
  • empirical software engineering
  • programming
  • recruitment
  • usable privacy and security

Fingerprint

Dive into the research topics of 'Recruiting Participants With Programming Skills: A Comparison of Four Crowdsourcing Platforms and a CS Student Mailing List'. Together they form a unique fingerprint.

Cite this