Methods We applied the Respiratory Index of Severity in Children (RISC) for HIV-negative children, the RISC-Malawi, and the Pneumonia Etiolo-gy Research for Child Health (PERCH) scores to hospitalized children in the Pneumonia REsearch Partnerships to Assess WHO REcommendations (PREPARE) data set. The PREPARE data set includes pooled data from 41 studies on pediatric pneumonia from across the world. We calculated test characteristics and the area under the curve (AUC) for each of these clin-ical prediction rules.
Results The RISC score for HIV-negative children was applied to 3574 children 0-24 months and demonstrated poor discriminatory abili-ty (AUC = 0.66, 95% confidence interval (CI) = 0.58-0.73) in the identi-fication of children at risk of hospitalized pneumonia-related mortality. The RISC-Malawi score had fair discriminatory value (AUC = 0.75, 95% CI = 0.74-0.77) among 17 864 children 2-59 months. The PERCH score was applied to 732 children 1-59 months and also demonstrated poor dis-criminatory value (AUC = 0.55, 95% CI = 0.37-0.73).
Conclusions In a large external application of the RISC, RISC-Malawi, and PERCH scores, a substantial number of children were misclassified for their risk of hospitalized pneumonia-related mortality. Although pneumonia risk scores have performed well among the cohorts in which they were derived, their performance diminished when externally applied. A generalizable risk assessment tool with higher sensitivity and specificity to identify children at risk of hospitalized pneumonia-related mortality may be needed. Such a generalizable risk assessment tool would need context-specific validation prior to implementation in that setting