Rolf H H Groenwold1, Karel G M Moons2, Romin Pajouheshnia3, Doug G Altman4, Gary S Collins4, Thomas P A Debray2, Johannes B Reitsma2, Richard D Riley5, Linda M Peelen3. 1. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, PO Box 85500, 3508 GA Utrecht, The Netherlands. Electronic address: r.h.h.groenwold@umcutrecht.nl. 2. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, PO Box 85500, 3508 GA Utrecht, The Netherlands; Dutch Cochrane Center, University Medical Center Utrecht, PO Box 85500, Utrecht, 3508 GA, The Netherlands. 3. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, PO Box 85500, 3508 GA Utrecht, The Netherlands. 4. Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, United Kingdom. 5. Research Institute for Primary Care and Health Sciences, Keele University, Staffordshire ST5 5BG, United Kingdom.
Abstract
OBJECTIVES: To compare different methods to handle treatment when developing a prognostic model that aims to produce accurate probabilities of the outcome of individuals if left untreated. STUDY DESIGN AND SETTING: Simulations were performed based on two normally distributed predictors, a binary outcome, and a binary treatment, mimicking a randomized trial or an observational study. Comparison was made between simply ignoring treatment (SIT), restricting the analytical data set to untreated individuals (AUT), inverse probability weighting (IPW), and explicit modeling of treatment (MT). Methods were compared in terms of predictive performance of the model and the proportion of incorrect treatment decisions. RESULTS: Omitting a genuine predictor of the outcome from the prognostic model decreased model performance, in both an observational study and a randomized trial. In randomized trials, the proportion of incorrect treatment decisions was smaller when applying AUT or MT, compared to SIT and IPW. In observational studies, MT was superior to all other methods regarding the proportion of incorrect treatment decisions. CONCLUSION: If a prognostic model aims to produce correct probabilities of the outcome in the absence of treatment, ignoring treatments that affect that outcome can lead to suboptimal model performance and incorrect treatment decisions. Explicitly, modeling treatment is recommended.
OBJECTIVES: To compare different methods to handle treatment when developing a prognostic model that aims to produce accurate probabilities of the outcome of individuals if left untreated. STUDY DESIGN AND SETTING: Simulations were performed based on two normally distributed predictors, a binary outcome, and a binary treatment, mimicking a randomized trial or an observational study. Comparison was made between simply ignoring treatment (SIT), restricting the analytical data set to untreated individuals (AUT), inverse probability weighting (IPW), and explicit modeling of treatment (MT). Methods were compared in terms of predictive performance of the model and the proportion of incorrect treatment decisions. RESULTS: Omitting a genuine predictor of the outcome from the prognostic model decreased model performance, in both an observational study and a randomized trial. In randomized trials, the proportion of incorrect treatment decisions was smaller when applying AUT or MT, compared to SIT and IPW. In observational studies, MT was superior to all other methods regarding the proportion of incorrect treatment decisions. CONCLUSION: If a prognostic model aims to produce correct probabilities of the outcome in the absence of treatment, ignoring treatments that affect that outcome can lead to suboptimal model performance and incorrect treatment decisions. Explicitly, modeling treatment is recommended.
Authors: Marc Lipman; Mahdad Noursadeghi; Ibrahim Abubakar; Rishi K Gupta; Claire J Calderwood; Alexei Yavlinsky; Maria Krutikov; Matteo Quartagno; Maximilian C Aichelburg; Neus Altet; Roland Diel; Claudia C Dobler; Jose Dominguez; Joseph S Doyle; Connie Erkens; Steffen Geis; Pranabashis Haldar; Anja M Hauri; Thomas Hermansen; James C Johnston; Christoph Lange; Berit Lange; Frank van Leth; Laura Muñoz; Christine Roder; Kamila Romanowski; David Roth; Martina Sester; Rosa Sloot; Giovanni Sotgiu; Gerrit Woltmann; Takashi Yoshiyama; Jean-Pierre Zellweger; Dominik Zenner; Robert W Aldridge; Andrew Copas; Molebogeng X Rangaka Journal: Nat Med Date: 2020-10-19 Impact factor: 53.440
Authors: Meron M Kifle; Prabin Dahal; Manu Vatish; Ana Sofia Cerdeira; Eric O Ohuma Journal: BMC Pregnancy Childbirth Date: 2022-06-27 Impact factor: 3.105
Authors: David M Kent; David van Klaveren; Jessica K Paulus; Ralph D'Agostino; Steve Goodman; Rodney Hayward; John P A Ioannidis; Bray Patrick-Lake; Sally Morton; Michael Pencina; Gowri Raman; Joseph S Ross; Harry P Selker; Ravi Varadhan; Andrew Vickers; John B Wong; Ewout W Steyerberg Journal: Ann Intern Med Date: 2019-11-12 Impact factor: 25.391
Authors: Zhe Xu; Matthew Arnold; David Stevens; Stephen Kaptoge; Lisa Pennells; Michael J Sweeting; Jessica Barrett; Emanuele Di Angelantonio; Angela M Wood Journal: Am J Epidemiol Date: 2021-10-01 Impact factor: 5.363
Authors: Romin Pajouheshnia; Linda M Peelen; Karel G M Moons; Johannes B Reitsma; Rolf H H Groenwold Journal: BMC Med Res Methodol Date: 2017-07-14 Impact factor: 4.615
Authors: Andrew S Moriarty; Lewis W Paton; Kym I E Snell; Richard D Riley; Joshua E J Buckman; Simon Gilbody; Carolyn A Chew-Graham; Shehzad Ali; Stephen Pilling; Nick Meader; Bob Phillips; Peter A Coventry; Jaime Delgadillo; David A Richards; Chris Salisbury; Dean McMillan Journal: Diagn Progn Res Date: 2021-07-02