TY - JOUR
T1 - External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database
T2 - individual participant data meta-analysis
AU - the IPPIC Collaborative Network
AU - Allotey, J.
AU - Whittle, R.
AU - Snell, K. I.E.
AU - Smuk, M.
AU - Townsend, R.
AU - von Dadelszen, P.
AU - Heazell, A. E.P.
AU - Magee, L.
AU - Smith, G. C.S.
AU - Sandall, J.
AU - Thilaganathan, B.
AU - Zamora, J.
AU - Riley, R. D.
AU - Khalil, A.
AU - Thangaratinam, S.
AU - Coomarasamy, Arri
AU - Kwong, Alex
AU - Savitri, Ary I.
AU - Salvesen, Kjell åsmund
AU - Bhattacharya, Sohinee
AU - Uiterwaal, Cuno S.P.M.
AU - Staff, Annetine C.
AU - Andersen, Louise Bjoerkholt
AU - Olive, Elisa Llurba
AU - Redman, Christopher
AU - Sletner, Line
AU - Daskalakis, George
AU - Macleod, Maureen
AU - Thilaganathan, Baskaran
AU - Abdollahain, Mali
AU - Ramírez, Javier Arenas
AU - Massé, Jacques
AU - Khalil, Asma
AU - Audibert, Francois
AU - Magnus, Per Minor
AU - Jenum, Anne Karen
AU - Baschat, Ahmet
AU - Ohkuchi, Akihide
AU - McAuliffe, Fionnuala M.
AU - West, Jane
AU - Askie, Lisa M.
AU - Mone, Fionnuala
AU - Farrar, Diane
AU - Zimmerman, Peter A.
AU - Smits, Luc J.M.
AU - Riddell, Catherine
AU - Kingdom, John C.
AU - van de Post, Joris
AU - Illanes, Sebastián E.
AU - Holzman, Claudia
AU - van Kuijk, Sander M.J.
AU - Carbillon, Lionel
AU - Villa, Pia M.
AU - Eskild, Anne
AU - Chappell, Lucy
AU - Prefumo, Federico
AU - Velauthar, Luxmi
AU - Seed, Paul
AU - van Oostwaard, Miriam
AU - Verlohren, Stefan
AU - Poston, Lucilla
AU - Ferrazzi, Enrico
AU - Vinter, Christina A.
AU - Nagata, Chie
AU - Brown, Mark
AU - Vollebregt, Karlijn C.
AU - Takeda, Satoru
AU - Langenveld, Josje
AU - Widmer, Mariana
AU - Saito, Shigeru
AU - Haavaldsen, Camilla
AU - Carroli, Guillermo
AU - Olsen, Jørn
AU - Wolf, Hans
AU - Zavaleta, Nelly
AU - Eisensee, Inge
AU - Vergani, Patrizia
AU - Lumbiganon, Pisake
AU - Makrides, Maria
AU - Facchinetti, Fabio
AU - Sequeira, Evan
AU - Gibson, Robert
AU - Ferrazzani, Sergio
AU - Frusca, Tiziana
AU - Norman, Jane E.
AU - Figueiró, Ernesto A.
AU - Lapaire, Olav
AU - Laivuori, Hannele
AU - Lykke, Jacob A.
AU - Conde-Agudelo, Agustin
AU - Galindo, Alberto
AU - Mbah, Alfred
AU - Betran, Ana Pilar
AU - Herraiz, Ignacio
AU - Trogstad, Lill
AU - Smith, Gordon G.S.
AU - Steegers, Eric A.P.
AU - Salim, Read
AU - Huang, Tianhua
AU - Adank, Annemarijne
AU - Zhang, Jun
AU - Meschino, Wendy S.
AU - Browne, Joyce L.
AU - Allen, Rebecca E.
AU - Costa, Fabricio Da Silva
AU - Klipstein-Grobusch Browne, Kerstin
AU - Crowther, Caroline A.
AU - Jørgensen, Jan Stener
AU - Forest, Jean Claude
AU - Rumbold, Alice R.
AU - Mol, Ben W.
AU - Giguère, Yves
AU - Kenny, Louise C.
AU - Ganzevoort, Wessel
AU - Odibo, Anthony O.
AU - Myers, Jenny
AU - Yeo, Seon Ae
AU - Goffinet, Francois
AU - McCowan, Lesley
AU - Pajkrt, Eva
AU - Teede, Helena J.
AU - Haddad, Bassam G.
AU - Dekker, Gustaaf
AU - Kleinrouweler, Emily C.
AU - LeCarpentier, Édouard
AU - Roberts, Claire T.
AU - Groen, Henk
AU - Skråstad, Ragnhild Bergene
AU - Heinonen, Seppo
AU - Eero, Kajantie
AU - Anggraini, Dewi
AU - Souka, Athena
AU - Cecatti, Jose Guilherme
AU - Monterio, Ilza
AU - Pillalis, Athanasios
AU - Souza, Renato
AU - Hawkins, Lee Ann
AU - Gabbay-Benziv, Rinat
AU - Crovetto, Francesca
AU - Figuera, Francesc
AU - Jorgensen, Laura
AU - Dodds, Julie
AU - Patel, Mehali
AU - Aviram, Amir
AU - Papageorghiou, Aris
AU - Khan, Khalid
N1 - Funding Information:
The IPPIC data repository was set up by funding from the National Institute for Health Research Health Technology Assessment Programme (Ref no: 14/158/02). This project was funded by Sands charity. K.I.E.S. is funded by the National Institute for Health Research School for Primary Care Research (NIHR SPCR Launching Fellowship).
Funding Information:
The UK Medical Research Council and Wellcome (grant ref: 102215/2/13/2) and the University of Bristol, Bristol, UK, provide core support for ALSPAC. This publication is the work of the authors and J.A., S.T., R.R. and R.W. will serve as guarantors for the contents of this paper.
Publisher Copyright:
© 2021 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of International Society of Ultrasound in Obstetrics and Gynecology.
PY - 2022/2/1
Y1 - 2022/2/1
N2 - Objective: Stillbirth is a potentially preventable complication of pregnancy. Identifying women at high risk of stillbirth can guide decisions on the need for closer surveillance and timing of delivery in order to prevent fetal death. Prognostic models have been developed to predict the risk of stillbirth, but none has yet been validated externally. In this study, we externally validated published prediction models for stillbirth using individual participant data (IPD) meta-analysis to assess their predictive performance. Methods: MEDLINE, EMBASE, DH-DATA and AMED databases were searched from inception to December 2020 to identify studies reporting stillbirth prediction models. Studies that developed or updated prediction models for stillbirth for use at any time during pregnancy were included. IPD from cohorts within the International Prediction of Pregnancy Complications (IPPIC) Network were used to validate externally the identified prediction models whose individual variables were available in the IPD. The risk of bias of the models and cohorts was assessed using the Prediction study Risk Of Bias ASsessment Tool (PROBAST). The discriminative performance of the models was evaluated using the C-statistic, and calibration was assessed using calibration plots, calibration slope and calibration-in-the-large. Performance measures were estimated separately in each cohort, as well as summarized across cohorts using random-effects meta-analysis. Clinical utility was assessed using net benefit. Results: Seventeen studies reporting the development of 40 prognostic models for stillbirth were identified. None of the models had been previously validated externally, and the full model equation was reported for only one-fifth (20%, 8/40) of the models. External validation was possible for three of these models, using IPD from 19 cohorts (491 201 pregnant women) within the IPPIC Network database. Based on evaluation of the model development studies, all three models had an overall high risk of bias, according to PROBAST. In the IPD meta-analysis, the models had summary C-statistics ranging from 0.53 to 0.65 and summary calibration slopes ranging from 0.40 to 0.88, with risk predictions that were generally too extreme compared with the observed risks. The models had little to no clinical utility, as assessed by net benefit. However, there remained uncertainty in the performance of some models due to small available sample sizes. Conclusions: The three validated stillbirth prediction models showed generally poor and uncertain predictive performance in new data, with limited evidence to support their clinical application. The findings suggest methodological shortcomings in their development, including overfitting. Further research is needed to further validate these and other models, identify stronger prognostic factors and develop more robust prediction models.
AB - Objective: Stillbirth is a potentially preventable complication of pregnancy. Identifying women at high risk of stillbirth can guide decisions on the need for closer surveillance and timing of delivery in order to prevent fetal death. Prognostic models have been developed to predict the risk of stillbirth, but none has yet been validated externally. In this study, we externally validated published prediction models for stillbirth using individual participant data (IPD) meta-analysis to assess their predictive performance. Methods: MEDLINE, EMBASE, DH-DATA and AMED databases were searched from inception to December 2020 to identify studies reporting stillbirth prediction models. Studies that developed or updated prediction models for stillbirth for use at any time during pregnancy were included. IPD from cohorts within the International Prediction of Pregnancy Complications (IPPIC) Network were used to validate externally the identified prediction models whose individual variables were available in the IPD. The risk of bias of the models and cohorts was assessed using the Prediction study Risk Of Bias ASsessment Tool (PROBAST). The discriminative performance of the models was evaluated using the C-statistic, and calibration was assessed using calibration plots, calibration slope and calibration-in-the-large. Performance measures were estimated separately in each cohort, as well as summarized across cohorts using random-effects meta-analysis. Clinical utility was assessed using net benefit. Results: Seventeen studies reporting the development of 40 prognostic models for stillbirth were identified. None of the models had been previously validated externally, and the full model equation was reported for only one-fifth (20%, 8/40) of the models. External validation was possible for three of these models, using IPD from 19 cohorts (491 201 pregnant women) within the IPPIC Network database. Based on evaluation of the model development studies, all three models had an overall high risk of bias, according to PROBAST. In the IPD meta-analysis, the models had summary C-statistics ranging from 0.53 to 0.65 and summary calibration slopes ranging from 0.40 to 0.88, with risk predictions that were generally too extreme compared with the observed risks. The models had little to no clinical utility, as assessed by net benefit. However, there remained uncertainty in the performance of some models due to small available sample sizes. Conclusions: The three validated stillbirth prediction models showed generally poor and uncertain predictive performance in new data, with limited evidence to support their clinical application. The findings suggest methodological shortcomings in their development, including overfitting. Further research is needed to further validate these and other models, identify stronger prognostic factors and develop more robust prediction models.
UR - http://www.scopus.com/inward/record.url?scp=85122335569&partnerID=8YFLogxK
U2 - 10.1002/uog.23757
DO - 10.1002/uog.23757
M3 - Article
C2 - 34405928
AN - SCOPUS:85122335569
SN - 0960-7692
VL - 59
SP - 209
EP - 219
JO - Ultrasound in Obstetrics & Gynecology
JF - Ultrasound in Obstetrics & Gynecology
IS - 2
ER -