External validation of the QCovid 2 and 3 risk prediction algorithms for risk of COVID-19 hospitalisation and mortality in adults: a national cohort study in Scotland

Objective: The QCovid 2 and 3 algorithms are risk prediction tools developed during the second wave of the COVID-19 pandemic that can be used to predict risk of COVID-19 hospitalisation and mortality, taking vaccination status into account. In this paper we assess their performance in Scotland.

Methods: We used the EAVE II national COVID-19 data platform consisting of individual-level data for the population of Scotland (5.4 million residents). Primary care data were linked to reverse-transcription polymerase chain reaction (RT-PCR) virology testing, hospitalisation and mortality data. We assessed the discrimination and calibration of the QCovid 2 and 3 algorithms in predicting COVID-19 hospitalisations and deaths between 8 December 2020 and 15 June 2021.

Results: Our validation dataset comprised 465,058 individuals, aged 19-100. We found the following performance metrics (95% confidence intervals) for QCovid 2 and 3: Harrell’s C 0.84 (0.82, 0.86) for hospitalisation, and 0.92 (0.90, 0.94) for death, observed-expected ratio of 0.24 for hospitalisation and 0.26 for death (i.e. both the number of hospitalisations and the number of deaths were overestimated), and a Brier score of 0.0009 (0.00084, 0.00096) for hospitalisation and 0.00036 (0.00032, 0.0004) for death.

Conclusions: We found good discrimination of the QCovid 2 and 3 algorithms in Scotland, although performance was worse in higher age groups. Both the number of hospitalisations and the number of deaths were overestimated.