Nina P Paynter1, Nancy R Cook1. 1. Division of Preventive Medicine, Brigham and Women’s Hospital, Boston, Massachusetts, USA (NPP and NRC)
Abstract
BACKGROUND: Comparing prediction models using reclassification within subgroups at intermediate risk is often of clinical interest. OBJECTIVE: To demonstrate a method for obtaining an unbiased estimate for the Net Reclassification Improvement (NRI) evaluated only on a subset, the clinical NRI. STUDY DESIGN: and Setting. We derived the expected value of the clinical NRI under the null hypothesis using the same principles as the overall NRI. We then conducted a simulation study based on a logistic model with a known predictor and a potential predictor, varying the effects of the known and potential predictors to test the performance of our bias-corrected clinical NRI measure. Finally, data from the Women's Health Study, a prospective cohort of 24 171 female health professionals, were used as an example of the proposed method. RESULTS: Our bias-corrected estimate is shown to have a mean of zero in the null case under a range of simulated parameters and, unlike the naïve estimate, to be unbiased. We also provide 2 methods for obtaining a variance estimate, both with reasonable type 1 errors. CONCLUSION: Our proposed method is an improvement over currently used methods of calculating the clinical NRI and is recommended to reduce overly optimistic results.
BACKGROUND: Comparing prediction models using reclassification within subgroups at intermediate risk is often of clinical interest. OBJECTIVE: To demonstrate a method for obtaining an unbiased estimate for the Net Reclassification Improvement (NRI) evaluated only on a subset, the clinical NRI. STUDY DESIGN: and Setting. We derived the expected value of the clinical NRI under the null hypothesis using the same principles as the overall NRI. We then conducted a simulation study based on a logistic model with a known predictor and a potential predictor, varying the effects of the known and potential predictors to test the performance of our bias-corrected clinical NRI measure. Finally, data from the Women's Health Study, a prospective cohort of 24 171 female health professionals, were used as an example of the proposed method. RESULTS: Our bias-corrected estimate is shown to have a mean of zero in the null case under a range of simulated parameters and, unlike the naïve estimate, to be unbiased. We also provide 2 methods for obtaining a variance estimate, both with reasonable type 1 errors. CONCLUSION: Our proposed method is an improvement over currently used methods of calculating the clinical NRI and is recommended to reduce overly optimistic results.
Authors: Paul M Ridker; Nancy R Cook; I-Min Lee; David Gordon; J Michael Gaziano; Joann E Manson; Charles H Hennekens; Julie E Buring Journal: N Engl J Med Date: 2005-03-07 Impact factor: 91.245
Authors: Samuli Ripatti; Emmi Tikkanen; Marju Orho-Melander; Aki S Havulinna; Kaisa Silander; Amitabh Sharma; Candace Guiducci; Markus Perola; Antti Jula; Juha Sinisalo; Marja-Liisa Lokki; Markku S Nieminen; Olle Melander; Veikko Salomaa; Leena Peltonen; Sekar Kathiresan Journal: Lancet Date: 2010-10-23 Impact factor: 79.321
Authors: Matthew E Mealiffe; Renee P Stokowski; Brian K Rhees; Ross L Prentice; Mary Pettinger; David A Hinds Journal: J Natl Cancer Inst Date: 2010-10-18 Impact factor: 13.506
Authors: Laine E Thomas; Emily C O'Brien; Jonathan P Piccini; Ralph B D'Agostino; Michael J Pencina Journal: Eur Heart J Date: 2019-06-14 Impact factor: 29.983
Authors: Michael G Shlipak; Kunihiro Matsushita; Johan Ärnlöv; Lesley A Inker; Ronit Katz; Kevan R Polkinghorne; Dietrich Rothenbacher; Mark J Sarnak; Brad C Astor; Josef Coresh; Andrew S Levey; Ron T Gansevoort Journal: N Engl J Med Date: 2013-09-05 Impact factor: 91.245
Authors: Jehu S Mathew; Michael C Sachs; Ronit Katz; Kristen K Patton; Susan R Heckbert; Andrew N Hoofnagle; Alvaro Alonso; Michel Chonchol; Rajat Deo; Joachim H Ix; David S Siscovick; Bryan Kestenbaum; Ian H de Boer Journal: Circulation Date: 2014-06-11 Impact factor: 29.690
Authors: F G R Fowkes; G D Murray; I Butcher; A R Folsom; A T Hirsch; D J Couper; G Debacker; M Kornitzer; A B Newman; K C Sutton-Tyrrell; M Cushman; A J Lee; J F Price; R B D'Agostino; J M Murabito; Pe Norman; K H Masaki; L M Bouter; R J Heine; C D A Stehouwer; M M McDermott; H E J H Stoffers; J A Knottnerus; M Ogren; B Hedblad; W Koenig; C Meisinger; J A Cauley; Oh Franco; M G M Hunink; A Hofman; J C Witteman; M H Criqui; R D Langer; W R Hiatt; R F Hamman Journal: Eur J Prev Cardiol Date: 2013-12-23 Impact factor: 7.804