K G M Moons1, A Rogier T Donders, E W Steyerberg, F E Harrell. 1. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 80035, 3508 TA Utrecht, The Netherlands. K.G.M.Moons@umcutrecht.nl
Abstract
BACKGROUND AND OBJECTIVE: There is growing interest in developing prediction models. The accuracy of such models when applied in new patient samples is commonly lower than estimated from the development sample. This may be because of differences between the samples and/or because the developed model was overfitted (too optimistic). Various methods, including bootstrapping techniques exist for afterwards shrinking the regression coefficients and the model's discrimination and calibration for overoptimism. Penalized maximum likelihood estimation (PMLE) is a more rigorous method because adjustment for overfitting is directly built into the model development, instead of relying on shrinkage afterwards. PMLE has been described mainly in the statistical literature and is rarely applied to empirical data. Using empirical data, we illustrate the use of PMLE to develop a prediction model. METHODS: The accuracy of the final PMLE model will be contrasted with the final models derived by ordinary stepwise logistic regression without and with shrinkage afterwards. The potential advantages and disadvantages of PMLE over the other two strategies are discussed. RESULTS: PMLE leads to smaller prediction errors, provides for model reduction to a user-defined degree, and may differently shrink each predictor for overoptimism without sacrificing much discriminative accuracy of the model. CONCLUSION: PMLE is an easily applicable and promising method to directly adjust clinical prediction models for overoptimism.
BACKGROUND AND OBJECTIVE: There is growing interest in developing prediction models. The accuracy of such models when applied in new patient samples is commonly lower than estimated from the development sample. This may be because of differences between the samples and/or because the developed model was overfitted (too optimistic). Various methods, including bootstrapping techniques exist for afterwards shrinking the regression coefficients and the model's discrimination and calibration for overoptimism. Penalized maximum likelihood estimation (PMLE) is a more rigorous method because adjustment for overfitting is directly built into the model development, instead of relying on shrinkage afterwards. PMLE has been described mainly in the statistical literature and is rarely applied to empirical data. Using empirical data, we illustrate the use of PMLE to develop a prediction model. METHODS: The accuracy of the final PMLE model will be contrasted with the final models derived by ordinary stepwise logistic regression without and with shrinkage afterwards. The potential advantages and disadvantages of PMLE over the other two strategies are discussed. RESULTS: PMLE leads to smaller prediction errors, provides for model reduction to a user-defined degree, and may differently shrink each predictor for overoptimism without sacrificing much discriminative accuracy of the model. CONCLUSION: PMLE is an easily applicable and promising method to directly adjust clinical prediction models for overoptimism.
Authors: Tyler W Barrett; Amy R Martin; Alan B Storrow; Cathy A Jenkins; Frank E Harrell; Stephan Russ; Dan M Roden; Dawood Darbar Journal: Ann Emerg Med Date: 2010-08-21 Impact factor: 5.721
Authors: Lori A Dolan; Stuart L Weinstein; Mark F Abel; Patrick P Bosch; Matthew B Dobbs; Tyler O Farber; Matthew F Halsey; M Timothy Hresko; Walter F Krengel; Charles T Mehlman; James O Sanders; Richard M Schwend; Suken A Shah; Kushagra Verma Journal: Spine Deform Date: 2019-11
Authors: M Puig-Asensio; M Fernández-Ruiz; J M Aguado; P Merino; D Lora-Pablos; J Guinea; P Martín-Dávila; M Cuenca-Estrella; B Almirante Journal: Antimicrob Agents Chemother Date: 2016-05-23 Impact factor: 5.191