Crowdsourcing for Information Visualization: Promises and Pitfalls

Rita Borgo; Bongshin Lee; Benjamin Bach; Sara Fabrikant; Radu  Jianu; Andreas Kerren; Stephen  Kobourov; Fintan  McGee; Luana Micallef; Tatiana von Landesberger; Katrin Ballweg; Stephan  Diehl; Paolo Simonetto; Michelle Zhou

doi:10.1007/978-3-319-66435-4_5

Crowdsourcing for Information Visualization: Promises and Pitfalls

Rita Borgo, Bongshin Lee, Benjamin Bach, Sara Fabrikant, Radu Jianu, Andreas Kerren, Stephen Kobourov, Fintan McGee, Luana Micallef, Tatiana von Landesberger, Katrin Ballweg, Stephan Diehl, Paolo Simonetto, Michelle Zhou

Research output: Chapter in Book/Report/Conference proceeding › Chapter

Abstract / Description of output

Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

Original language	English
Title of host publication	Evaluation in the Crowd: Crowdsourcing and Human-Centered Experiments
Publisher	Springer
Number of pages	43
ISBN (Electronic)	978-3-319-66435-4
ISBN (Print)	978-3-319-66434-7
DOIs	https://doi.org/10.1007/978-3-319-66435-4_5
Publication status	Published - 28 Sept 2017

Access to Document

10.1007/978-3-319-66435-4_5

https://link.springer.com/chapter/10.1007%2F978-3-319-66435-4_5

Cite this

Borgo, R., Lee, B., Bach, B., Fabrikant, S., Jianu, R., Kerren, A., Kobourov, S., McGee, F., Micallef, L., von Landesberger, T., Ballweg, K., Diehl, S., Simonetto, P., & Zhou, M. (2017). Crowdsourcing for Information Visualization: Promises and Pitfalls. In Evaluation in the Crowd: Crowdsourcing and Human-Centered Experiments Springer. https://doi.org/10.1007/978-3-319-66435-4_5

@inbook{44060f41ac244355a3fe79dc1b920467,

title = "Crowdsourcing for Information Visualization: Promises and Pitfalls",

abstract = "Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.",

author = "Rita Borgo and Bongshin Lee and Benjamin Bach and Sara Fabrikant and Radu Jianu and Andreas Kerren and Stephen Kobourov and Fintan McGee and Luana Micallef and {von Landesberger}, Tatiana and Katrin Ballweg and Stephan Diehl and Paolo Simonetto and Michelle Zhou",

year = "2017",

month = sep,

day = "28",

doi = "10.1007/978-3-319-66435-4_5",

language = "English",

isbn = "978-3-319-66434-7",

booktitle = "Evaluation in the Crowd: Crowdsourcing and Human-Centered Experiments",

publisher = "Springer",

}

TY - CHAP

T1 - Crowdsourcing for Information Visualization: Promises and Pitfalls

AU - Borgo, Rita

AU - Lee, Bongshin

AU - Bach, Benjamin

AU - Fabrikant, Sara

AU - Jianu, Radu

AU - Kerren, Andreas

AU - Kobourov, Stephen

AU - McGee, Fintan

AU - Micallef, Luana

AU - von Landesberger, Tatiana

AU - Ballweg, Katrin

AU - Diehl, Stephan

AU - Simonetto, Paolo

AU - Zhou, Michelle

PY - 2017/9/28

Y1 - 2017/9/28

N2 - Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

AB - Crowdsourcing offers great potential to overcome the limitations of controlled lab studies. To guide future designs of crowdsourcing-based studies for visualization, we review visualization research that has attempted to leverage crowdsourcing for empirical evaluations of visualizations. We discuss six core aspects for successful employment of crowdsourcing in empirical studies for visualization – participants, study design, study procedure, data, tasks, and metrics & measures. We then present four case studies, discussing potential mechanisms to overcome common pitfalls. This chapter will help the visualization community understand how to effectively and efficiently take advantage of the exciting potential crowdsourcing has to offer to support empirical visualization research.

U2 - 10.1007/978-3-319-66435-4_5

DO - 10.1007/978-3-319-66435-4_5

M3 - Chapter

SN - 978-3-319-66434-7

BT - Evaluation in the Crowd: Crowdsourcing and Human-Centered Experiments

PB - Springer

ER -

Crowdsourcing for Information Visualization: Promises and Pitfalls

Abstract / Description of output

Access to Document

Fingerprint

Cite this