Setting weights for fifteen CHNRI criteria at the global and regional level using public stakeholders: An Amazon Mechanical Turk study

Kerri Wazny, John Ravenscroft, Kit Chan, Diego G. Bassani, Niall Anderson, Igor Rudan

Research output: Contribution to journalArticlepeer-review


Introduction: Stakeholder involvement has been described as an indispensable part of health research priority setting. Yet, more than 75% of the exercises using the Child Health and Nutrition Research Initiative (CHNRI) methodology have omitted the step involving stakeholders in priority setting. Those that have used stakeholders have not used the public, possibly due to the difficulty of assembling and/or accessing a public stakeholder group. In order to strengthen future exercises using the CHNRI methodology, we have used a public stakeholder group to weight 15 CHNRI criteria, and to explore regional differences and whether being a health stakeholder is influential, and whether the criteria are collapsible.
Methods: Using Amazon Mechanical Turk (AMT), an online crowdsourcing platform, we collected demographic information and conducted a Likert-scale format survey about the importance of the CHNRI criteria from 1,051 stakeholders. The Kruskal-Wallis test, with Dunn’s test for posthoc comparisons, was used to examine regional differences and Wilcoxon rank-sum test was used to analyse differences between stakeholders with health training/background and stakeholders without a health background and by region. A Factor Analysis (FA) was conducted on the criteria to identify the main domains connecting them. Criteria means were converted to weights.
Results: There were regional differences in thirteen of fifteen criteria according to the Kruskal-Wallis test and differences in responses from health stakeholders versus those who were not in eleven of fifteen criteria using the Wilcoxon rank-sum test. Three components were identified: improve and impact results; implementation and affordability; and, study design and dissemination. A formula is provided to convert means to weights for future studies.
Conclusion: In future CHNRI studies, researchers will need to ensure adequate representation from stakeholders to undue bias of CHNRI results. These results should be used in combination with other stakeholder groups, including government, donors, policy makers, and bilateral agencies. Global and regional stakeholder groups scored CHNRI criteria differently; due to this, researchers should consider which group to use in their CHNRI exercises.
Original languageEnglish
Article number010702
Pages (from-to)1-11
Number of pages11
JournalJournal of Global Health
Issue number1
Early online date2 Apr 2019
Publication statusPublished - 1 Jun 2019


  • Adolescent
  • Adult
  • Aged
  • Biomedical Research/organization & administration
  • Child Health
  • Crowdsourcing
  • Female
  • Global Health
  • Health Priorities
  • Humans
  • India
  • Male
  • Middle Aged
  • Stakeholder Participation
  • United States
  • Young Adult


Dive into the research topics of 'Setting weights for fifteen CHNRI criteria at the global and regional level using public stakeholders: An Amazon Mechanical Turk study'. Together they form a unique fingerprint.

Cite this