Jay M Brahmbhatt1, Travis Hee Wai2, Christopher H Goss3, Erika D Lease3, Christian A Merlo4, Siddhartha G Kapnadak3, Kathleen J Ramos5. 1. Division of General Internal Medicine, Department of Medicine, University of Washington, Seattle, Washington. 2. Department of Biostatistics, University of Washington, Seattle, Washington. 3. Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle, Washington. 4. Johns Hopkins University School of Medicine, Division of Pulmonary and Critical Care, Baltimore, Maryland. 5. Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle, Washington. Electronic address: ramoskj@uw.edu.
Abstract
BACKGROUND: Improved predictive models are needed in lung transplantation in the setting of a proposed allocation system that incorporates longer-term post-transplant survival in the United States. Allocation systems require accurate mortality predictions to justly allocate organs. METHODS: Utilizing the United Network for Organ Sharing database (2005-2017), we fit models to predict 1-year mortality based on the Lung Allocation Score (LAS), the Chan, et al, 2019 model, a novel "clinician" model (a priori clinician selection of pre-transplant covariates), and two machine learning models (Least Absolute Shrinkage and Selection Operator; LASSO and Random Forests) for predicting 1-year and 3-year post-transplant mortality. We compared predictive accuracy among models. We evaluated the calibration of models by comparing average predicted probability vs observed outcome per decile. We repeated analyses fit for 3-year mortality, disease category, including donor covariates, and LAS era. RESULTS: The area under the cure for all models was low, ranging from 0.55 to 0.62. All exhibited reasonable negative predictive values (0.87-0.90), but the positive predictive value for was poor (all <0.25). Evaluating LAS calibration found 1-year post-transplant estimates consistently overestimated risk of mortality, with greater differences in higher deciles. LASSO, Random Forests, and clinician models showed no improvement when evaluated by disease category or with the addition of donor covariates and performed worse for 3-year outcomes. CONCLUSIONS: The LAS overestimated patients' risk of post-transplant death, thus underestimating transplant benefit in the sickest candidates. Novel models based on pre-transplant recipient covariates failed to improve prediction. There should be wariness in post-transplant survival predictions from available models.
BACKGROUND: Improved predictive models are needed in lung transplantation in the setting of a proposed allocation system that incorporates longer-term post-transplant survival in the United States. Allocation systems require accurate mortality predictions to justly allocate organs. METHODS: Utilizing the United Network for Organ Sharing database (2005-2017), we fit models to predict 1-year mortality based on the Lung Allocation Score (LAS), the Chan, et al, 2019 model, a novel "clinician" model (a priori clinician selection of pre-transplant covariates), and two machine learning models (Least Absolute Shrinkage and Selection Operator; LASSO and Random Forests) for predicting 1-year and 3-year post-transplant mortality. We compared predictive accuracy among models. We evaluated the calibration of models by comparing average predicted probability vs observed outcome per decile. We repeated analyses fit for 3-year mortality, disease category, including donor covariates, and LAS era. RESULTS: The area under the cure for all models was low, ranging from 0.55 to 0.62. All exhibited reasonable negative predictive values (0.87-0.90), but the positive predictive value for was poor (all <0.25). Evaluating LAS calibration found 1-year post-transplant estimates consistently overestimated risk of mortality, with greater differences in higher deciles. LASSO, Random Forests, and clinician models showed no improvement when evaluated by disease category or with the addition of donor covariates and performed worse for 3-year outcomes. CONCLUSIONS: The LAS overestimated patients' risk of post-transplant death, thus underestimating transplant benefit in the sickest candidates. Novel models based on pre-transplant recipient covariates failed to improve prediction. There should be wariness in post-transplant survival predictions from available models.
Authors: Mark J Russo; Berhane Worku; Alexander Iribarne; Kimberly N Hong; Jonathan A Yang; Wickii Vigneswaran; Joshua R Sonett Journal: J Thorac Cardiovasc Surg Date: 2011-05 Impact factor: 5.209
Authors: Cynthia J Gries; Tessa C Rue; Patrick J Heagerty; Jeffrey D Edelman; Michael S Mulligan; Christopher H Goss Journal: J Heart Lung Transplant Date: 2010-04-09 Impact factor: 10.247
Authors: Jonathan P Singer; Eric R Peterson; Mark E Snyder; Patricia P Katz; Jeffrey A Golden; Frank D'Ovidio; Matthew Bacchetta; Joshua R Sonett; Jasleen Kukreja; Lori Shah; Hilary Robbins; Kristin Van Horn; Rupal J Shah; Joshua M Diamond; Nancy Wickersham; Li Sun; Steven Hays; Selim M Arcasoy; Scott M Palmer; Lorraine B Ware; Jason D Christie; David J Lederer Journal: Am J Respir Crit Care Med Date: 2014-11-01 Impact factor: 21.405
Authors: Satish Chandrashekaran; Cesar A Keller; Walter K Kremers; Steve G Peters; Matthew A Hathcock; Cassie C Kennedy Journal: J Heart Lung Transplant Date: 2014-11-17 Impact factor: 10.247
Authors: T M Egan; S Murray; R T Bustami; T H Shearon; K P McCullough; L B Edwards; M A Coke; E R Garrity; S C Sweet; D A Heiney; F L Grover Journal: Am J Transplant Date: 2006 Impact factor: 8.086
Authors: Patricia J Rodriguez; David L Veenstra; Patrick J Heagerty; Christopher H Goss; Kathleen J Ramos; Aasthaa Bansal Journal: Value Health Date: 2021-12-22 Impact factor: 5.101
Authors: B G Maxwell; J E Levitt; B A Goldstein; J J Mooney; M R Nicolls; M Zamora; V Valentine; D Weill; G S Dhillon Journal: Am J Transplant Date: 2014-09-10 Impact factor: 8.086