Daan Nieboer1, Yvonne Vergouwe2, Monique J Roobol3, Donna P Ankerst4, Michael W Kattan5, Andrew J Vickers6, Ewout W Steyerberg2. 1. Department of Public Health, Erasmus MC-University Medical Center Rotterdam, Rotterdam, The Netherlands. Electronic address: d.nieboer@erasmusmc.nl. 2. Department of Public Health, Erasmus MC-University Medical Center Rotterdam, Rotterdam, The Netherlands. 3. Department of Urology, Erasmus MC-University Medical Center Rotterdam, Rotterdam, The Netherlands. 4. Department of Mathematics, Technical University Munich, Munich, Germany; University of Texas Health Science Center at San Antonio, San Antonio, TX, USA. 5. Cleveland Clinic Foundation Quantitative Health Sciences Division, Cleveland, OH, USA. 6. Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York, USA.
Abstract
OBJECTIVES: We aimed to compare nonlinear modeling methods for handling continuous predictors for reproducibility and transportability of prediction models. STUDY DESIGN AND SETTING: We analyzed four cohorts of previously unscreened men who underwent prostate biopsy for diagnosing prostate cancer. Continuous predictors of prostate cancer included prostate-specific antigen and prostate volume. The logistic regression models included linear terms, logarithmic terms, fractional polynomials of degree one or two (FP1 and FP2), or restricted cubic splines (RCS) with three or five knots (RCS3 and RCS5). The resulting models were internally validated by bootstrap resampling and externally validated in the cohorts not used at model development. Performance was assessed with the area under the receiver operating characteristic curve (AUC) and the calibration component of the Brier score (CAL). RESULTS: At internal validation models with FP2 or RCS5 showed slightly better performance than the other models (typically 0.004 difference in AUC and 0.001 in CAL). At external validation models containing logarithms, FP1, or RCS3 showed better performance (differences 0.01 and 0.002). CONCLUSION: Flexible nonlinear modeling methods led to better model performance at internal validation. However, when application of the model is intended across a wide range of settings, less flexible functions may be more appropriate to maximize external validity.
OBJECTIVES: We aimed to compare nonlinear modeling methods for handling continuous predictors for reproducibility and transportability of prediction models. STUDY DESIGN AND SETTING: We analyzed four cohorts of previously unscreened men who underwent prostate biopsy for diagnosing prostate cancer. Continuous predictors of prostate cancer included prostate-specific antigen and prostate volume. The logistic regression models included linear terms, logarithmic terms, fractional polynomials of degree one or two (FP1 and FP2), or restricted cubic splines (RCS) with three or five knots (RCS3 and RCS5). The resulting models were internally validated by bootstrap resampling and externally validated in the cohorts not used at model development. Performance was assessed with the area under the receiver operating characteristic curve (AUC) and the calibration component of the Brier score (CAL). RESULTS: At internal validation models with FP2 or RCS5 showed slightly better performance than the other models (typically 0.004 difference in AUC and 0.001 in CAL). At external validation models containing logarithms, FP1, or RCS3 showed better performance (differences 0.01 and 0.002). CONCLUSION: Flexible nonlinear modeling methods led to better model performance at internal validation. However, when application of the model is intended across a wide range of settings, less flexible functions may be more appropriate to maximize external validity.
Authors: E W Steyerberg; F E Harrell; G J Borsboom; M J Eijkemans; Y Vergouwe; J D Habbema Journal: J Clin Epidemiol Date: 2001-08 Impact factor: 6.437
Authors: Ian M Thompson; Donna Pauler Ankerst; Chen Chi; Phyllis J Goodman; Catherine M Tangen; M Scott Lucia; Ziding Feng; Howard L Parnes; Charles A Coltman Journal: J Natl Cancer Inst Date: 2006-04-19 Impact factor: 13.506
Authors: Usha S Govindarajulu; Donna Spiegelman; Sally W Thurston; Bhaswati Ganguli; Ellen A Eisen Journal: Stat Med Date: 2007-09-10 Impact factor: 2.373
Authors: Thomas P A Debray; Yvonne Vergouwe; Hendrik Koffijberg; Daan Nieboer; Ewout W Steyerberg; Karel G M Moons Journal: J Clin Epidemiol Date: 2014-08-30 Impact factor: 6.437
Authors: Fritz H Schröder; Jonas Hugosson; Monique J Roobol; Teuvo L J Tammela; Stefano Ciatto; Vera Nelen; Maciej Kwiatkowski; Marcos Lujan; Hans Lilja; Marco Zappa; Louis J Denis; Franz Recker; Antonio Berenguer; Liisa Määttänen; Chris H Bangma; Gunnar Aus; Arnauld Villers; Xavier Rebillard; Theodorus van der Kwast; Bert G Blijenberg; Sue M Moss; Harry J de Koning; Anssi Auvinen Journal: N Engl J Med Date: 2009-03-18 Impact factor: 91.245
Authors: Georg Bartsch; Wolfgang Horninger; Helmut Klocker; Alexandre Pelzer; Jasmin Bektic; Wilhelm Oberaigner; Harald Schennach; Georg Schäfer; Ferdinand Frauscher; Mathieu Boniol; Gianluca Severi; Chris Robertson; Peter Boyle Journal: BJU Int Date: 2008-04 Impact factor: 5.588
Authors: Andrew G McIntosh; Daniel C Parker; Brian L Egleston; Robert G Uzzo; Mohammed Haseebuddin; Shreyas S Joshi; Rosalia Viterbo; Richard E Greenberg; David Y T Chen; Marc C Smaldone; Alexander Kutikov Journal: BJU Int Date: 2019-06-30 Impact factor: 5.588
Authors: Gary S Collins; Emmanuel O Ogundimu; Jonathan A Cook; Yannick Le Manach; Douglas G Altman Journal: Stat Med Date: 2016-05-18 Impact factor: 2.373
Authors: Thomas Willigenburg; Marieke J van Son; Sandrine M G van de Pol; Wietse S C Eppinga; Jan J W Lagendijk; Hans C J de Boer; Marinus A Moerland; Jochem R N van der Voort van Zyp; Max Peters Journal: Clin Transl Radiat Oncol Date: 2021-06-29