Ben Van Calster1, Laure Wynants2, Jan F M Verbeek3, Jan Y Verbakel4, Evangelia Christodoulou2, Andrew J Vickers5, Monique J Roobol3, Ewout W Steyerberg6. 1. Department of Development and Regeneration, KU Leuven, Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands. Electronic address: Ben.vancalster@kuleuven.be. 2. Department of Development and Regeneration, KU Leuven, Leuven, Belgium. 3. Department of Urology, Erasmus MC, Rotterdam, The Netherlands. 4. Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium; Nuffield Department of Primary Care Health Sciences, University of Oxford, UK. 5. Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA. 6. Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands; Department of Public Health, Erasmus MC, Rotterdam, The Netherlands.
Abstract
CONTEXT: Urologists regularly develop clinical risk prediction models to support clinical decisions. In contrast to traditional performance measures, decision curve analysis (DCA) can assess the utility of models for decision making. DCA plots net benefit (NB) at a range of clinically reasonable risk thresholds. OBJECTIVE: To provide recommendations on interpreting and reporting DCA when evaluating prediction models. EVIDENCE ACQUISITION: We informally reviewed the urological literature to determine investigators' understanding of DCA. To illustrate, we use data from 3616 patients to develop risk models for high-grade prostate cancer (n=313, 9%) to decide who should undergo a biopsy. The baseline model includes prostate-specific antigen and digital rectal examination; the extended model adds two predictors based on transrectal ultrasound (TRUS). EVIDENCE SYNTHESIS: We explain risk thresholds, NB, default strategies (treat all, treat no one), and test tradeoff. To use DCA, first determine whether a model is superior to all other strategies across the range of reasonable risk thresholds. If so, that model appears to improve decisions irrespective of threshold. Second, consider if there are important extra costs to using the model. If so, obtain the test tradeoff to check whether the increase in NB versus the best other strategy is worth the additional cost. In our case study, addition of TRUS improved NB by 0.0114, equivalent to 1.1 more detected high-grade prostate cancers per 100 patients. Hence, adding TRUS would be worthwhile if we accept subjecting 88 patients to TRUS to find one additional high-grade prostate cancer or, alternatively, subjecting 10 patients to TRUS to avoid one unnecessary biopsy. CONCLUSIONS: The proposed guidelines can help researchers understand DCA and improve application and reporting. PATIENT SUMMARY: Decision curve analysis can identify risk models that can help us make better clinical decisions. We illustrate appropriate reporting and interpretation of decision curve analysis.
CONTEXT: Urologists regularly develop clinical risk prediction models to support clinical decisions. In contrast to traditional performance measures, decision curve analysis (DCA) can assess the utility of models for decision making. DCA plots net benefit (NB) at a range of clinically reasonable risk thresholds. OBJECTIVE: To provide recommendations on interpreting and reporting DCA when evaluating prediction models. EVIDENCE ACQUISITION: We informally reviewed the urological literature to determine investigators' understanding of DCA. To illustrate, we use data from 3616 patients to develop risk models for high-grade prostate cancer (n=313, 9%) to decide who should undergo a biopsy. The baseline model includes prostate-specific antigen and digital rectal examination; the extended model adds two predictors based on transrectal ultrasound (TRUS). EVIDENCE SYNTHESIS: We explain risk thresholds, NB, default strategies (treat all, treat no one), and test tradeoff. To use DCA, first determine whether a model is superior to all other strategies across the range of reasonable risk thresholds. If so, that model appears to improve decisions irrespective of threshold. Second, consider if there are important extra costs to using the model. If so, obtain the test tradeoff to check whether the increase in NB versus the best other strategy is worth the additional cost. In our case study, addition of TRUS improved NB by 0.0114, equivalent to 1.1 more detected high-grade prostate cancers per 100 patients. Hence, adding TRUS would be worthwhile if we accept subjecting 88 patients to TRUS to find one additional high-grade prostate cancer or, alternatively, subjecting 10 patients to TRUS to avoid one unnecessary biopsy. CONCLUSIONS: The proposed guidelines can help researchers understand DCA and improve application and reporting. PATIENT SUMMARY: Decision curve analysis can identify risk models that can help us make better clinical decisions. We illustrate appropriate reporting and interpretation of decision curve analysis.
Authors: Monique J Roobol; Heidi A van Vugt; Stacy Loeb; Xiaoye Zhu; Meelan Bul; Chris H Bangma; Arno G L J H van Leenders; Ewout W Steyerberg; Fritz H Schröder Journal: Eur Urol Date: 2011-11-15 Impact factor: 20.096
Authors: Ewout W Steyerberg; Andrew J Vickers; Nancy R Cook; Thomas Gerds; Mithat Gonen; Nancy Obuchowski; Michael J Pencina; Michael W Kattan Journal: Epidemiology Date: 2010-01 Impact factor: 4.822
Authors: Ben Van Calster; Andrew J Vickers; Michael J Pencina; Stuart G Baker; Dirk Timmerman; Ewout W Steyerberg Journal: Med Decis Making Date: 2013-01-11 Impact factor: 2.583
Authors: Anwar R Padhani; Jelle Barentsz; Geert Villeirs; Andrew B Rosenkrantz; Daniel J Margolis; Baris Turkbey; Harriet C Thoeny; François Cornud; Masoom A Haider; Katarzyna J Macura; Clare M Tempany; Sadhna Verma; Jeffrey C Weinreb Journal: Radiology Date: 2019-06-11 Impact factor: 11.105
Authors: Kelly L Zuromski; Berk Ustun; Irving Hwang; Terence M Keane; Brian P Marx; Murray B Stein; Robert J Ursano; Ronald C Kessler Journal: Depress Anxiety Date: 2019-07-29 Impact factor: 6.505
Authors: Tyler J Loftus; Amanda C Filiberto; Yanjun Li; Jeremy Balch; Allyson C Cook; Patrick J Tighe; Philip A Efron; Gilbert R Upchurch; Parisa Rashidi; Xiaolin Li; Azra Bihorac Journal: Surgery Date: 2020-06-13 Impact factor: 3.982