Ype de Jong1, Chava L Ramspek2, Vera H W van der Endt2, Maarten B Rookmaaker3, Peter J Blankestijn3, Robin W M Vernooij4, Marianne C Verhaar3, Willem Jan W Bos5, Friedo W Dekker2, Gurbey Ocak3, Merel van Diepen2. 1. Department of Clinical Epidemiology, Leiden University Medical Center (LUMC), Leiden, The Netherlands; Department of Internal Medicine, Leiden University Medical Center (LUMC), Leiden, The Netherlands. Electronic address: Y.de_jong@lumc.nl. 2. Department of Clinical Epidemiology, Leiden University Medical Center (LUMC), Leiden, The Netherlands. 3. Department of Nephrology and Hypertension, University Medical Center Utrecht (UMCU), Utrecht, The Netherlands. 4. Department of Nephrology and Hypertension, University Medical Center Utrecht (UMCU), Utrecht, The Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands. 5. Department of Internal Medicine, Leiden University Medical Center (LUMC), Leiden, The Netherlands; Department of Internal Medicine, Sint Antonius Hospital, Nieuwegein, The Netherlands.
Abstract
OBJECTIVES: The objective of this study was to systematically review and externally assess the predictive performance of models for ischemic stroke in incident dialysis patients. STUDY DESIGN AND SETTING: Two reviewers systematically searched and selected ischemic stroke models. Risk of bias was assessed with the PROBAST. Predictive performance was evaluated within The Netherlands Cooperative Study on the Adequacy of Dialysis (NECOSAD), a large prospective multicenter cohort of incident dialysis patients. For discrimination, c-statistics were calculated; calibration was assessed by plotting predicted and observed probabilities for stroke, and calibration-in-the-large. RESULTS: Seventy-seven prediction models for stroke were identified, of which 15 were validated. Risk of bias was high, with all of these models scoring high risk in one or more domains. In NECOSAD, of the 1,955 patients, 127 (6.5%) suffered an ischemic stroke during the follow-up of 2.5 years. Compared with the original studies, most models performed worse with all models showing poor calibration and discriminative abilities (c-statistics ranging from 0.49 to 0.66). The Framingham showed reasonable calibration; however, with a c-statistic of 0.57 (95% CI 0.50-0.63), the discrimination was poor. CONCLUSION: This external validation demonstrates the weak predictive performance of ischemic stroke models in incident dialysis patients. Instead of using these models in this fragile population, either existing models should be updated, or novel models should be developed and validated.
OBJECTIVES: The objective of this study was to systematically review and externally assess the predictive performance of models for ischemic stroke in incident dialysis patients. STUDY DESIGN AND SETTING: Two reviewers systematically searched and selected ischemic stroke models. Risk of bias was assessed with the PROBAST. Predictive performance was evaluated within The Netherlands Cooperative Study on the Adequacy of Dialysis (NECOSAD), a large prospective multicenter cohort of incident dialysis patients. For discrimination, c-statistics were calculated; calibration was assessed by plotting predicted and observed probabilities for stroke, and calibration-in-the-large. RESULTS: Seventy-seven prediction models for stroke were identified, of which 15 were validated. Risk of bias was high, with all of these models scoring high risk in one or more domains. In NECOSAD, of the 1,955 patients, 127 (6.5%) suffered an ischemic stroke during the follow-up of 2.5 years. Compared with the original studies, most models performed worse with all models showing poor calibration and discriminative abilities (c-statistics ranging from 0.49 to 0.66). The Framingham showed reasonable calibration; however, with a c-statistic of 0.57 (95% CI 0.50-0.63), the discrimination was poor. CONCLUSION: This external validation demonstrates the weak predictive performance of ischemic stroke models in incident dialysis patients. Instead of using these models in this fragile population, either existing models should be updated, or novel models should be developed and validated.
Authors: Chava L Ramspek; Lucy Teece; Kym I E Snell; Marie Evans; Richard D Riley; Maarten van Smeden; Nan van Geloven; Merel van Diepen Journal: Int J Epidemiol Date: 2022-05-09 Impact factor: 9.685
Authors: Ype de Jong; Edouard L Fu; Merel van Diepen; Marco Trevisan; Karolina Szummer; Friedo W Dekker; Juan J Carrero; Gurbey Ocak Journal: Eur Heart J Date: 2021-04-14 Impact factor: 29.983
Authors: Ype de Jong; Esmee M van der Willik; Jet Milders; Yvette Meuleman; Rachael L Morton; Friedo W Dekker; Merel van Diepen Journal: BMC Nephrol Date: 2021-09-13 Impact factor: 2.388
Authors: Gurbey Ocak; Meriem Khairoun; Othman Khairoun; Willem Jan W Bos; Edouard L Fu; Maarten J Cramer; Jan Westerink; Marianne C Verhaar; Frank L Visseren Journal: PLoS One Date: 2022-04-07 Impact factor: 3.240
Authors: Gurbey Ocak; Mark Roest; Marianne C Verhaar; Maarten B Rookmaaker; Peter J Blankestijn; Willem Jan W Bos; Rob Fijnheer; Nathalie C Péquériaux; Friedo W Dekker Journal: BMC Nephrol Date: 2021-06-16 Impact factor: 2.388