US Studies May Overestimate Effect Sizes in Softer Research

Daniele Fanelli, John P A Ioannidis

Research output: Contribution to journalArticlepeer-review

Abstract

Many biases affect scientific research, causing a waste of resources, posing a threat to human health, and hampering scientific progress. These problems are hypothesized to be worsened by lack of consensus on theories and methods, by selective publication processes, and by career systems too heavily oriented toward productivity, such as those adopted in the United States (US). Here, we extracted 1,174 primary outcomes appearing in 82 meta-analyses published in health-related biological and behavioral research sampled from the Web of Science categories Genetics & Heredity and Psychiatry and measured how individual results deviated from the overall summary effect size within their respective meta-analysis. We found that primary studies whose outcome included behavioral parameters were generally more likely to report extreme effects, and those with a corresponding author based in the US were more likely to deviate in the direction predicted by their experimental hypotheses, particularly when their outcome did not include additional biological parameters. Nonbehavioral studies showed no such "US effect" and were subject mainly to sampling variance and small-study effects, which were stronger for non-US countries. Although this latter finding could be interpreted as a publication bias against non-US authors, the US effect observed in behavioral research is unlikely to be generated by editorial biases. Behavioral studies have lower methodological consensus and higher noise, making US researchers potentially more likely to express an underlying propensity to report strong and significant findings.
Original languageEnglish
Pages (from-to)15031-15036
Number of pages6
JournalProceedings of the National Academy of Sciences
Volume110
Issue number37
DOIs
Publication statusPublished - 10 Sep 2013

Fingerprint

Dive into the research topics of 'US Studies May Overestimate Effect Sizes in Softer Research'. Together they form a unique fingerprint.

Cite this