Hatem Al-Farra1, Ameen Abu-Hanna2, Bas A J M de Mol3, W J Ter Burg2, Saskia Houterman4, José P S Henriques3, Anita C J Ravelli2. 1. Department of Medical Informatics, Amsterdam UMC - Location AMC, University of Amsterdam, the Netherlands; Heart Center, Amsterdam UMC - Location AMC, University of Amsterdam, Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands. Electronic address: h.alfarra@amsterdamumc.nl. 2. Department of Medical Informatics, Amsterdam UMC - Location AMC, University of Amsterdam, the Netherlands. 3. Heart Center, Amsterdam UMC - Location AMC, University of Amsterdam, Amsterdam Cardiovascular Sciences, Amsterdam, the Netherlands. 4. The Netherlands Heart Registration, Utrecht, the Netherlands.
Abstract
BACKGROUND: Several mortality prediction models (MPM) are used for predicting early (30-day) mortality following transcatheter aortic valve implantation (TAVI). Little is known about their predictive performance in external TAVI populations. We aim to externally validate established MPMs on a large TAVI dataset from the Netherlands Heart Registration (NHR). METHODS: We included data from NHR-patients who underwent TAVI during 2013-2017. We calculated the predicted mortalities per MPM. We assessed the predictive performance by discrimination (Area Under Receiver Operating-characteristic Curve, AU-ROC); the Area Under Precision-Recall Curve, AU-PRC; calibration (using calibration-intercept and calibration-slope); Brier Score and Brier Skill Score. We also assessed the predictive performance among subgroups: tertiles of mortality-risk for non-survivors, gender, and access-route. RESULTS: We included 6177 TAVI-patients with an observed early-mortality rate of 4.5% (n = 280). We applied seven MPMs (STS, EuroSCORE-I, EuroSCORE-II, ACC-TAVI, FRANCE-2, OBSERVANT, and German-AV) on our cohort. The highest AU-ROCs were 0.64 (95%CI 0.61-0.67) for ACC-TAVI and 0.63 (95%CI 0.60-0.67) for FRANCE-2. All MPMs had a very low AU-PRC of ≤0.09. ACC-TAVI had the best calibration-intercept and calibration-slope. Brier Score values ranged between 0.043 and 0.063. Brier Skill Score ranged between -0.47 and 0.004. ACC-TAVI and FRANCE-2 predicted high mortality-risk better than other MPMs. ACC-TAVI outperformed other MPMs in different subgroups. CONCLUSION: The ACC-TAVI model has relatively the best predictive performance. However, all models have poor predictive performance. Because of the poor discrimination, miscalibration and limited accuracy of the models there is a need to update the existing models or develop new TAVI-specific models for local populations.
BACKGROUND: Several mortality prediction models (MPM) are used for predicting early (30-day) mortality following transcatheter aortic valve implantation (TAVI). Little is known about their predictive performance in external TAVI populations. We aim to externally validate established MPMs on a large TAVI dataset from the Netherlands Heart Registration (NHR). METHODS: We included data from NHR-patients who underwent TAVI during 2013-2017. We calculated the predicted mortalities per MPM. We assessed the predictive performance by discrimination (Area Under Receiver Operating-characteristic Curve, AU-ROC); the Area Under Precision-Recall Curve, AU-PRC; calibration (using calibration-intercept and calibration-slope); Brier Score and Brier Skill Score. We also assessed the predictive performance among subgroups: tertiles of mortality-risk for non-survivors, gender, and access-route. RESULTS: We included 6177 TAVI-patients with an observed early-mortality rate of 4.5% (n = 280). We applied seven MPMs (STS, EuroSCORE-I, EuroSCORE-II, ACC-TAVI, FRANCE-2, OBSERVANT, and German-AV) on our cohort. The highest AU-ROCs were 0.64 (95%CI 0.61-0.67) for ACC-TAVI and 0.63 (95%CI 0.60-0.67) for FRANCE-2. All MPMs had a very low AU-PRC of ≤0.09. ACC-TAVI had the best calibration-intercept and calibration-slope. Brier Score values ranged between 0.043 and 0.063. Brier Skill Score ranged between -0.47 and 0.004. ACC-TAVI and FRANCE-2 predicted high mortality-risk better than other MPMs. ACC-TAVI outperformed other MPMs in different subgroups. CONCLUSION: The ACC-TAVI model has relatively the best predictive performance. However, all models have poor predictive performance. Because of the poor discrimination, miscalibration and limited accuracy of the models there is a need to update the existing models or develop new TAVI-specific models for local populations.
Authors: Hatem Al-Farra; Bas A J M de Mol; Anita C J Ravelli; W J P P Ter Burg; Saskia Houterman; José P S Henriques; Ameen Abu-Hanna; M M Vis; J Vos; L Timmers; W A L Tonino; C E Schotborgh; V Roolvink; F Porta; M G Stoel; S Kats; G Amoroso; H W van der Werf; P R Stella; P de Jaegere Journal: Int J Cardiol Heart Vasc Date: 2021-01-23