Non-Standard Errors

Albert J. Menkveld, Anna Dreber, Felix Holzmeister, Juergen Huber, Magnus Johannesson, Michael Kirchler, Sebastian Neususs, Michael Razen, Utz Weitzel

Research output: Working paper

Abstract / Description of output

In statistics, samples are drawn from a population in a data generating process (DGP). Standard errors measure the uncertainty in sample estimates of population parameters. In science,evidence is generated to test hypotheses in an evidence generating process (EGP). We claim that EGP variation across researchers adds uncertainty: non-standard errors. To study them, we let 164 teams test six hypotheses on the same sample.We find that non-standard errors are sizeable, on par with standard errors. Their size (i) co-varies only weakly with team merits, reproducibility, or peer rating, (ii) declines significantly after peer-feedback, and (iii) is underestimated by participants.
Original languageEnglish
Number of pages50
Publication statusPublished - 26 Nov 2021


Dive into the research topics of 'Non-Standard Errors'. Together they form a unique fingerprint.

Cite this