Relationship between quality and payment in crowdsourced design

Hao Wu, Jonathan Corney, Michael Grant, Jiang-Liang Hou (Editor)

Research output: Other contribution

Abstract

In recent years, the “power of the crowd” has been repeatedly demonstrated and various Internet platforms have been used to applied collaborative intelligence to areas that range from open innovation to image analysis. However, crowdsourcing applications in the fields of design research and creative innovation have been much slower to emerge. So, although there have been reports of systems and researchers using Internet crowdsourcing to carry out generative design, there are still many gaps in knowledge about the capability and limitations of the technology. For example on crowdsourcing platforms, like Amazon’s Mechanical Turk, the relationship between remuneration and the final quality of designs has not been established, so it is unclear how much payment should be offered to ensure a particular standard of result. To investigate the relationship between crowd’s remuneration and the quality of their innovation, this paper reports how payment for a 2D interior design task (living room layout) was systematically varied and the quality of the output assessed by a ranking process designed that was also crowdsourced. Information about individual Mturk workers who participated in the study was also collected. The results suggest that while that average design quality only slowly increases, the quality of the “best” design generated by the crowd improved dramatically with payment levels. In other words, increasing monetary rewards does not improve the average creativity of design but rather it increases the chance of an excellent solution being generated by an individual in the crowd.
Original languageEnglish
PublisherIEEE Xplore
Number of pages6
Place of PublicationPiscataway, NJ.
ISBN (Print)9781479937769
DOIs
Publication statusPublished - 2014

Keywords

  • Amazon Mechanical Turk (MTurk)
  • crowdscourcing
  • crowdsourcing demographics
  • design productivity
  • interior design
  • open innovation
  • architectural design
  • innovation
  • interactive computer systems
  • internet
  • product design

Fingerprint

Dive into the research topics of 'Relationship between quality and payment in crowdsourced design'. Together they form a unique fingerprint.

Cite this