Patricia J Rodriguez1, David L Veenstra2, Patrick J Heagerty3, Christopher H Goss4, Kathleen J Ramos5, Aasthaa Bansal6. 1. The Comparative Health Outcomes, Policy & Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA. Electronic address: prodrig@uw.edu. 2. The Comparative Health Outcomes, Policy & Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA. 3. Department of Biostatistics, University of Washington, Seattle, WA, USA. 4. Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, University of Washington, Seattle, WA, USA; Division of Pulmonology, Department of Pediatrics, University of Washington, Seattle, WA, USA. 5. Division of Pulmonary, Critical Care and Sleep Medicine, Department of Medicine, University of Washington, Seattle, WA, USA. 6. The Comparative Health Outcomes, Policy & Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA. Electronic address: abansal@uw.edu.
Abstract
OBJECTIVES: We propose a framework of health outcomes modeling with dynamic decision making and real-world data (RWD) to evaluate the potential utility of novel risk prediction models in clinical practice. Lung transplant (LTx) referral decisions in cystic fibrosis offer a complex case study. METHODS: We used longitudinal RWD for a cohort of adults (n = 4247) from the Cystic Fibrosis Foundation Patient Registry to compare outcomes of an LTx referral policy based on machine learning (ML) mortality risk predictions to referral based on (1) forced expiratory volume in 1 second (FEV1) alone and (2) heterogenous usual care (UC). We then developed a patient-level simulation model to project number of patients referred for LTx and 5-year survival, accounting for transplant availability, organ allocation policy, and heterogenous treatment effects. RESULTS: Only 12% of patients (95% confidence interval 11%-13%) were referred for LTx over 5 years under UC, compared with 19% (18%-20%) under FEV1 and 20% (19%-22%) under ML. Of 309 patients who died before LTx referral under UC, 31% (27%-36%) would have been referred under FEV1 and 40% (35%-45%) would have been referred under ML. Given a fixed supply of organs, differences in referral time did not lead to significant differences in transplants, pretransplant or post-transplant deaths, or overall survival in 5 years. CONCLUSIONS: Health outcomes modeling with RWD may help to identify novel ML risk prediction models with high potential real-world clinical utility and rule out further investment in models that are unlikely to offer meaningful real-world benefits.
OBJECTIVES: We propose a framework of health outcomes modeling with dynamic decision making and real-world data (RWD) to evaluate the potential utility of novel risk prediction models in clinical practice. Lung transplant (LTx) referral decisions in cystic fibrosis offer a complex case study. METHODS: We used longitudinal RWD for a cohort of adults (n = 4247) from the Cystic Fibrosis Foundation Patient Registry to compare outcomes of an LTx referral policy based on machine learning (ML) mortality risk predictions to referral based on (1) forced expiratory volume in 1 second (FEV1) alone and (2) heterogenous usual care (UC). We then developed a patient-level simulation model to project number of patients referred for LTx and 5-year survival, accounting for transplant availability, organ allocation policy, and heterogenous treatment effects. RESULTS: Only 12% of patients (95% confidence interval 11%-13%) were referred for LTx over 5 years under UC, compared with 19% (18%-20%) under FEV1 and 20% (19%-22%) under ML. Of 309 patients who died before LTx referral under UC, 31% (27%-36%) would have been referred under FEV1 and 40% (35%-45%) would have been referred under ML. Given a fixed supply of organs, differences in referral time did not lead to significant differences in transplants, pretransplant or post-transplant deaths, or overall survival in 5 years. CONCLUSIONS: Health outcomes modeling with RWD may help to identify novel ML risk prediction models with high potential real-world clinical utility and rule out further investment in models that are unlikely to offer meaningful real-world benefits.
Authors: Karel G M Moons; Andre Pascal Kengne; Diederick E Grobbee; Patrick Royston; Yvonne Vergouwe; Douglas G Altman; Mark Woodward Journal: Heart Date: 2012-03-07 Impact factor: 5.994
Authors: Sebastian Vollmer; Bilal A Mateen; Gergo Bohner; Franz J Király; Rayid Ghani; Pall Jonsson; Sarah Cumbers; Adrian Jonas; Katherine S L McAllister; Puja Myles; David Granger; Mark Birse; Richard Branson; Karel G M Moons; Gary S Collins; John P A Ioannidis; Chris Holmes; Harry Hemingway Journal: BMJ Date: 2020-03-20
Authors: Kathleen J Ramos; Ranjani Somayaji; Erika D Lease; Christopher H Goss; Moira L Aitken Journal: BMC Pulm Med Date: 2017-01-19 Impact factor: 3.317
Authors: Anne L Stephenson; Kathleen J Ramos; Jenna Sykes; Xiayi Ma; Sanja Stanojevic; Bradley S Quon; Bruce C Marshall; Kristofer Petren; Joshua S Ostrenga; Aliza K Fink; Albert Faro; Alexander Elbert; Cecilia Chaparro; Christopher H Goss Journal: J Heart Lung Transplant Date: 2020-12-07 Impact factor: 10.247
Authors: Christopher J Kelly; Alan Karthikesalingam; Mustafa Suleyman; Greg Corrado; Dominic King Journal: BMC Med Date: 2019-10-29 Impact factor: 8.775
Authors: Jay M Brahmbhatt; Travis Hee Wai; Christopher H Goss; Erika D Lease; Christian A Merlo; Siddhartha G Kapnadak; Kathleen J Ramos Journal: J Heart Lung Transplant Date: 2022-05-20 Impact factor: 13.569