Arrhythmic risk prediction in arrhythmogenic right ventricular cardiomyopathy: external validation of the arrhythmogenic right ventricular cardiomyopathy risk calculator.

AIMS: Arrhythmogenic right ventricular cardiomyopathy (ARVC) causes ventricular arrhythmias (VAs) and sudden cardiac death (SCD). In 2019, a risk prediction model that estimates the 5-year risk of incident VAs in ARVC was developed (ARVCrisk.com). This study aimed to externally validate this prediction model in a large international multicentre cohort and to compare its performance with the risk factor approach recommended for implantable cardioverter-defibrillator (ICD) use by published guidelines and expert consensus. METHODS AND
RESULTS: In a retrospective cohort of 429 individuals from 29 centres in North America and Europe, 103 (24%) experienced sustained VA during a median follow-up of 5.02 (2.05-7.90) years following diagnosis of ARVC. External validation yielded good discrimination [C-index of 0.70 (95% confidence interval-CI 0.65-0.75)] and calibration slope of 1.01 (95% CI 0.99-1.03). Compared with the three published consensus-based decision algorithms for ICD use in ARVC (Heart Rhythm Society consensus on arrhythmogenic cardiomyopathy, International Task Force consensus statement on the treatment of ARVC, and American Heart Association guidelines for VA and SCD), the risk calculator performed better with a superior net clinical benefit below risk threshold of 35%.
CONCLUSION: Using a large independent cohort of patients, this study shows that the ARVC risk model provides good prognostic information and outperforms other published decision algorithms for ICD use. These findings support the use of the model to facilitate shared decision making regarding ICD implantation in the primary prevention of SCD in ARVC.

See the editorial comment for this article ‘Arrhythmogenic right ventricular cardiomyopathy: the never-ending quest for a risk calculator’, by Estelle Gandjbakhch and Annina S. Vischer,

Introduction

Arrhythmogenic right ventricular cardiomyopathy (ARVC) is a significant cause of sustained ventricular arrhythmia (VA) and sudden cardiac death (SCD), especially in young individuals and athletes. Preventing this catastrophic outcome through the prophylactic use of implantable cardioverter-defibrillators (ICDs) is a cornerstone of the disease management. Given the significant drawbacks associated with ICDs in this young and active population, appropriate patient selection is essential.

Over the past 25 years, numerous studies have identified predictors of sustained VA and SCD in ARVC and consensus documents have integrated these in decision algorithms for ICD use.[1-3] Building on this knowledge, a risk prediction model for sustained VA and SCD in ARVC was recently developed in a multinational cohort (n = 528, designed as the derivation cohort) mostly including high volume referral centres for ARVC.[4] This prediction model provides individualized prediction of the risk of VA in patients with ARVC without a prior history of sustained VA. Since its online publication, the risk calculator’s official site (http://www.ARVCrisk.com) has been used ∼20 000 times illustrating its uptake in clinical practice.

The model has been internally and externally validated in small studies.[4-9] However, adequately powered external validation is still lacking,[10] yet is paramount to confirm the reproducibility, generalizability, and need to update the model in an independent population.

The aims of the present study are thus (i) to conduct external validation of the published risk calculator in a distinct, adequately powered, and geographically diverse cohort including patients from six countries across North America and Europe and (ii) to compare the performance of the risk prediction model with other published guidelines and expert consensus recommendations for ICD use. During the current validation study, our group detected an inaccuracy in the formula of the original ARVC risk calculator published in 2019. It was corrected both on the website (ARVCrisk.com) and in the published manuscript.[11] We base the present study on the corrected risk calculator.

Methods

Study design

We conducted an observational, retrospective, longitudinal cohort study in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement.[12]

Study population

The study population was derived from 29 centres (see Supplementary material online, ) in six European and North American countries. This current cohort will be designated as the ‘validation cohort’ while the cohort leading to the published model will be designated as the ‘derivation cohort’. New patients from two centres participating in the original study (Montreal Heart Institute and Johns Hopkins Hospital) were included (52 patients; 12% of the cohort). No patients in the current cohort were included in the original ARVC derivation cohort. From each site, consistent with the derivation cohort, consecutive patients who (i) were diagnosed with definite ARVC as per 2010 Task Force Criteria (TFC),[13] (ii) were alive at presentation, and (iii) had not experienced spontaneous sustained VA or sudden cardiac arrest (SCA) at diagnosis were included. The study conforms to the Helsinki declaration and was approved by local ethics and/or institutional review boards. To maintain patient confidentiality, data, and study materials will not be made available to other researchers for purposes of replicating the results. A limited dataset may be made available upon request.

Data collection

Data were collected independently by each of the participating centres using uniform definitions. A complete list of variables and their definitions can be found in Supplementary material online, . Genetic variants were reviewed according to the American College of Medical Genetics and Genomics guidelines by cardiologists specialized in cardiovascular genetics (R.T. and J.C.T.).[14]

Missing data

Patients with >50% of predictors missing were excluded from the analysis. Missingness was assumed to be at random and imputed using multiple imputation by chained equations.[15] Missing quantitative values for right ventricular ejection fraction (RVEF) and left ventricular ejection fraction (LVEF) were imputed manually when only qualitative assessment was available as done previously[4] and detailed in Supplementary material online, . The multiple imputation model included all pre-specified predictors, proband status and genotype together with the outcome, and cumulative baseline hazard estimation.[16] A total of 25 imputed datasets were generated, and the final inference estimations were combined using Rubin’s rules.[17]

Study outcomes

In accordance with the published ARVC risk prediction model which this study aims to validate, the primary outcome was the first sustained VA following the definite diagnosis as per the TFC. Sustained VA was defined as a composite of the occurrence of SCD, SCA, spontaneous sustained ventricular tachycardia (VT; lasting ≥30 s at ≥100 b.p.m. or with haemodynamic compromise requiring cardioversion), ventricular fibrillation/flutter, or appropriate ICD intervention. Heart transplantation, cardiovascular mortality, and all-cause mortality were also collected.

Predictor variables and risk calculator

The same candidate predictors as those selected in the published model based on prior literature were considered.[18-20] These include sex, age, recent cardiac syncope (here defined as transient loss of consciousness and postural tone with spontaneous recovery with a likely arrhythmic mechanism, within a year of diagnosis), non-sustained VT (NSVT: defined as hemodynamically stable VT at ≥100 b.p.m., for ≥3 beats <30 s), number of premature ventricular complexes (PVCs) on 24-h Holter monitoring, the extent of T-wave inversion (TWI) on anterior and inferior leads, and RVEF. Each predictor variable was determined at the time of diagnosis, defined as 1 year before to 1 year after the date of diagnosis as per 2010 TFC and prior to the occurrence of the primary outcome.

The 5-year risk of sustained VA for an individual patient as per the published model is calculated using the following equation[4]:where the linear predictor (LP) was calculated according to the equation:Of note, the baseline hazard for 5-year prediction (0.8396) has been corrected since the initial publication in 2019.[11]

Statistical analysis

Analyses were performed with RStudio version 1.3.1093 (Boston, MA, USA). Continuous variables were expressed as mean ± standard deviation or median [interquartile range (IQR)] and compared using either the independent sample t-test or the Mann–Whitney U test. Categorical variables were presented as frequencies (%) and compared using the Fisher’s exact test. Follow-up duration was calculated as the time interval between the time of definite diagnosis according to TFC and the endpoint or censoring. Censoring was defined as death from any other cause, heart transplantation or the most recent follow-up visit at which the endpoint could be ascertained. Event-free survival probability was estimated using the Kaplan–Meier method and Cox proportional hazard regression analysis.

Model validation

The approach to external validation follows the method suggested by Royston and Altman for Cox prognostic models.[21] First, the overall discriminative performance of the model was measured using Harrell’s C-statistic, and the model fit by calculating the calibration slope, the regression of the LP (i.e. the product of the variable part of the Cox model) in the current cohort (validation cohort). Graphical evaluation of calibration was performed by plotting the predicted risk against the observed risk of sustained VA, using grouped Kaplan–Meier estimates and the continuous hazard regression function. The choice of the number of groups presented was based on the balance between providing sufficient spread in group risk, while maintaining adequate group sizes for precision. For the complete cohort, five groups are presented while for subgroup analyses, four groups are presented.

Subsequently, a more in-depth analysis of the model fit was performed by a Cox’s model including the same predictor variables in combination with the LP of the original model (as an offset variable) to evaluate potential differences in the regression coefficients of each individual predictor. The result indicating the validity of the model would be that if all coefficients ß* equalled 0, reflecting that all the variability in the validation sample is accounted for by the published model. In addition, the baseline survival function of the validation dataset was compared to that of the derivation dataset to see if the overall predictions need to be globally shifted upward or downward. Lastly, a new prediction model using the same predictor variables was fitted to the validation dataset and compared to the fit of the original model using the Akaike information criterion (AIC), with a difference of >2 defined as statistically significant. This allows testing whether a model specifically fitted to the validation dataset performs better than the original model in the validation dataset.

Subgroup analyses

We visually explored the performance of the model specifically in different populations of interest by comparing calibration plots for these subgroups. We stratified the cohort by geographic origin (Europe vs. North America), by proband status and by plakophilin 2 carrier status (PKP2; causal variant carrier vs. non-carrier). We did not report quantitative markers of performance such as the C-statistic as this study was not powered adequately for these subgroups.

To assess the impact of carrying an ICD on prediction accuracy, we also presented calibration plots based on ICD carrier status at baseline defined as ICD implantation prior to a year following diagnosis and first VA outcome, whichever came first.

Clinical utility

To assess the relative clinical utility of the risk prediction model, it was compared to three other published expert consensus algorithms for ICD implantation in ARVC: the 2015 International Task Force Consensus for the treatment of ARVC (ITFC),[18] the 2017 American Heart Association (AHA) guidelines for the management of VA and prevention of SCD,[2] and the 2019 Heart Rhythm Society (HRS) consensus on arrhythmogenic cardiomyopathy (excluding programmed ventricular stimulation)[22] through decision curve analysis. In a decision curve analysis,[23] the clinical benefit is assessed by the ‘net benefit’ representing the balance between useful (i.e. in patients with events) vs. useless (i.e. in patients without events) ICD placement at 5 years weighted according to the threshold used for ICD implantation. More specifically, the decision curve uses the following formula:where ‘pt’ represents threshold probability, in the current case, threshold for ICD implantation. Therefore, the higher the threshold used, the greater the harm of useless ICD use (i.e. false positive) is valued. Higher values indicate greater benefit while a value of 0 indicates no benefit.

To present the consequence of setting different thresholds for ICD implantation, we evaluated and plotted the proportion of patients who would receive ICDs and the proportion of treated and missed events at each threshold. We compared these with the recommendations for ICD use by the three published consensus mentioned above [ITFC(1), AHA (2), HRS(3)].

Results

The study population included 429 definite ARVC patients without a history of sustained VA or SCA at the time of diagnosis aged 43.1± 15.8 years and slightly more than half (n = 235, 54.8%) were male. Probands accounted for two-thirds of the cohort (n = 278, 64.8%). Half (n = 198, 46.6%) of patients had a pathogenic or likely pathogenic variant in a gene with definite or moderate association with ARVC,[24] which represents 70% (198 patients) of the 282 patients for whom the complete genetic information was available. PKP2 was the most common genotype, carried by 111 patients (26%) followed by DSP in 38 patients (9%). Compared to PKP2 patients, DSP patients were more likely to have a decrease in LVEF < 50% (44.7% vs. 6.4%) but less likely to have VA events at follow-up (13.2% vs. 24.5%). Baseline characteristics according to genotype are presented in Supplementary material online, . Other clinical and demographic characteristics are summarized in Table1. Baseline characteristics by country of origin are presented in Supplementary material online, , and a comparison of the derivation and validation cohort populations is presented in Supplementary material online, .

Table 1

Baseline clinical characteristics

	Overall (n = 429)	Non-sustained VA (n = 326)	Sustained VA (n = 103)	P-value
Demographics and genetics
Age at diagnosis (years)	43.1 ± 15.8	44.1 ± 15.7	40.1 ± 16.0	0.025
Male sex	235 (54.8)	159 (48.8)	76 (73.8)	<0.001
Proband status	278 (64.8)	197 (60.4)	81 (78.6)	0.001
(Likely) pathogenic variants (n = 282)	198 (46.2)	150 (46.0)	48 (46.6)	0.480
Genotype				0.302
PKP2	111 (25.6)	84 (25.8)	27 (26.2)
DSP	38 (8.9)	33 (10.1)	5 (4.9)
DSG2	27 (6.3)	22 (6.7)	5 (4.9)
DSC2	3 (0.7)	1 (0.3)	2 (1.9)
JUP	0 (0.0)	0 (0.0)	0 (0.0)
TMEM43	10 (2.3)	4 (1.2)	6 (5.8)
PLN	3 (0.7)	3 (0.9)	0 (0.0)
Multiple mutations	6 (1.4)	3 (0.9)	3 (2.9)
Clinical history
Recent cardiac syncope (n = 424)	37 (8.6)	16 (4.9)	21 (20.4)	<0.001
ECG/continuous ECG monitoring
TWI in ≥3 precordial leads (n = 409)	250 (58.3)	187 (57.4)	63 (61.2)	0.295
TWI in ≥2 inferior leads (n = 403)	109 (25.4)	81 (24.8)	28 (27.2)	0.589
PVC count (n = 324)	1434 (439–3601)	1354 (400–3719)	1676 (602–3492)	0.160
NSVT (n = 359)	148 (34.5)	105 (32.2)	43 (41.7)	0.001
Imaging
RVEF (%) (n = 410)	45 (36–53)	47 (38–53)	40 (35–48.5)	<0.001
LVEF (%) (n = 404)	57 (51–60)	57 (51–61)	57 (50–60)	0.049
Treatment at baseline
ICD	175 (40.8)	113 (34.7)	62 (60.2)	<0.001
Anti-arrhythmic drugs (n = 408)				0.041
Amiodarone	23 (6.0)	16 (4.9)	10 (9.8)
Sotalol	79 (18.4)	55 (16.9)	24 (23.3)
Propafenone/flecainide	15 (3.5)	9 (2.8)	6 (5.8)
β-blockers (n = 407)	206 (48.0)	156 (47.9)	50 (48.5)	0.50
Follow-up	5.02 (2.05–7.90)	4.48 (1.86–7.32)	6.12 (2.60–10.08)	0.002

Variables are expressed as frequency (%), mean ± standard deviation, or median (interquartile range). Total number of patients with available data for a given variable are mentioned in parenthesis for variables with missing data.

DSC2, desmocollin-2; DSG2, desmoglein-2; DSP, desmoplakin; ICD, implantable cardioverter-defibrillator; LVEF, left ventricular ejection fraction; NSVT, non-sustained ventricular tachycardia; JUP, junction plakoglobin; PKP2, plakophilin-2; PLN, phospholamban; PVC, premature ventricular complex; RVEF, right ventricular ejection fraction; TMEM43, transmembrane protein 43; TWI, T-wave inversion; VA, ventricular arrhythmia.

Overall, 299 (70.0%) patients had complete data for the pre-specified predictors. Six of the eight predictors had missing data: recent cardiac syncope (n = 5, 1.17%), NSVT (70 = 16.32%), PVC count (n = 105, 24.48%), extent of leads with TWI (n = 26, 6.06%) and RVEF (n = 19, 4.43%). From an initial cohort of 433 patients, four patients were excluded as >50% of their predictors were missing (four predictors or more).

Outcomes

During a median follow-up of [5.02 (2.05–7.90)] years, 103 patients (24%) experienced sustained VA events corresponding to an annual event rate of 4.98% [95% confidence interval (CI) 4.07–6.04]. shows the cumulative survival free from first sustained VA.

Survival free from sustained ventricular arrhythmia at follow-up. The cumulative event-free survival for any ventricular arrhythmia with 95% confidence intervals (shaded area) is plotted.

Among patients who experienced sustained VA during follow-up, the most common events were ICD treated VAs, which represented 59.2% of events (n = 61), followed by sustained VT (n = 32, 31.1%), SCA (n = 7, 6.8%), and SCD (n = 3, 2.9%). In patients with sustained or ICD treated VT events, the median cycle length (available in 57/93 events) was 280 ms (IQR: 246–315) which corresponds to 214 b.p.m. (190–243).

At last follow-up, 9 (2.1%) patients had died, including 2 from non-cardiac causes, and 7 (1.6%) had undergone heart transplantation.

External validation

Model validation revealed a Harrell C-index of 0.70 (95% CI 0.65–0.75). The calibration slope was 1.01 (95% CI 0.99–1.03) showing no significant difference in discrimination. The calibration of the model is graphically presented in demonstrating good overall agreement between predicted and observed shorter-term (1 year) and longer-term durations (5 year) with no significant over or under prediction across the complete risk spectrum. The distribution of patients according to their risk is presented in Supplementary material online, and calibration plots for intermediate durations (1, 2, 3, and 5 years) in Supplementary material online, .

Calibration plots presenting the agreement between predicted (x-axis) and observed (y-axis) 1-year (Panel A) and 5-year (Panel B) risk of ventricular arrhythmia. Triangles represent binned Kaplan–Meier estimates with 95% confidence intervals for quintiles of predicted risk. The straight line is the continuous calibration hazard regression with the dotted line represents optimal calibration (i.e. perfect correspondence between predictions and observations across the risk spectrum). The calibration is shown to be acceptable across the risk spectrum with no significant under or over prediction in any risk category. VA, ventricular arrhythmia.

Two different aspects of the model fit or potential misspecification were evaluated. First, the assessment of individual predictor coefficients () all showed no significant diversion from the original model in this cohort. This finding means that none of the individual coefficient would benefit from being modified from their original values to improve prediction in this cohort.

Assessment of the model fit. Assessment of the individual predictors (A) show an absence of diversion from the initial model as all coefficients are non-significantly different from 0. Compared survival probability of the derivation and validation cohorts (B) and baseline survival hazard (i.e. predictors-adjusted survival) presented as survival curves (C) both show similar expected survival. NSVT, non-sustained ventricular tachycardia; PVC, premature ventricular complex; TWI, T-wave inversion; RVEF, right ventricular ejection fraction.

Second, the baseline survival function (i.e. predictors-adjusted survival) was assessed through the comparison of the baseline survival probabilities (i.e. predictors-adjusted survival) in the derivation and the validation cohorts at different time points showing similar expected survival curves as shown numerically and visually in and . These findings suggest that the survival function does not need to be modified to improve prediction in this cohort.

Finally, the potential need to update the model was assessed by comparing the fit of the published model with the derivation of a new model in the validation cohort. The AIC of the published model in the current cohort (1059.14) and of a model derived in this cohort (1060.93) were not significantly different (absolute difference in AIC of 1.79) indicating the absence of significant improvement in predictions when fitting a model to this population.

As a sensitivity analysis, we repeated the process in patients with complete data (n = 299) resulting in a similar C-statistic, calibration slope, baseline risk, and calibration plot (see Supplementary material online, ).

Clinical utility

We compared the performance of the risk calculator with published consensus-based decision algorithms for ICD use in ARVC. As illustrated in , the risk calculator generally had a superior net clinical benefit when compared to the other published algorithms for ICD use. Its performance becomes similar to the HRS consensus above a risk of ∼35%.

Finally, we graphically presented the impact of different threshold for ICD implantation on the proportion of ICD use and the protection rate and compared to the published decision algorithms (). Higher thresholds result in less ICD use but less protection from VA. As an example, a threshold of 15% would results in implanting 59.4% of patients with ICDs while protecting 85.7% of patients with incident VA events.

Impact of implantable cardioverter-defibrillator use threshold on clinical outcomes. The potential impact of different thresholds for implantable cardioverter-defibrillator use according to the model is presented on the left side and the proportion of patients who would get an implantable cardioverter-defibrillator according to the different consensus statements is presented on the right side. For each threshold (x-axis) the proportion of patients (y-axis) who have events (red) who do not have events (blue), who would receive an implantable cardioverter-defibrillator (solid colours) or not receive one (hashed colours) are presented. The triangles represent the number of implantable cardioverter-defibrillator needed per event prevented for each threshold (right-sided y-axis). The numerical values are presented in the table below. Implantable cardioverter-defibrillator:ventricular arrhythmia, ratio of implantable cardioverter-defibrillator placements required to protect one patient developing ventricular arrhythmia; other abbreviations as in figure 4.

Figure 4

Decision curve analysis comparing the clinical utility of our model (dashed thick black line) with the 2015 International Task Force Consensus Statement algorithm for the treatment of arrhythmogenic right ventricular cardiomyopathy (dashed red line), the 2017 American Heart Association algorithm for the management of ventricular arrhythmia and prevention of sudden cardiac death (dashed green line) and the 2019 Heart Rhythm Society consensus on arrhythmogenic cardiomyopathy with exclusion of the Programmed ventricular stimulation (dashed blue line). The clinical utility of each treatment strategy is compared by plotting the net benefit (y-axis) for a range of possible implantable cardioverter-defibrillator placement thresholds based on the 5-year risk of ventricular arrhythmia (x-axis). Higher net benefit values indicate greater benefit while a value of 0 indicates no benefit. The published risk calculator depicted a better net benefit than the other published algorithms for implantable cardioverter-defibrillator implantation thresholds below a 35%. Above this threshold its performance was similar to the Heart Rhythm Society consensus algorithm. ICD, implantable cardioverter-defibrillator; ARVC, arrhythmogenic right ventricular cardiomyopathy, VA ventricular arrhythmia, SCD sudden cardiac death.

Subgroups analyses

The performance in subgroups of interest was visually explored by calibration plots presented in Supplementary material online, . This cohort was not sufficiently powered to provide definite answers in these subgroups.

Calibration appeared acceptable in patients from both Europe and North America, although this analysis had low precision in the North American population due to its smaller size.

The model performed well both in probands and family members with a possible trend toward overestimation in family members in the lower risk spectrum. The calibration was also visually acceptable both in PKP2 carriers and non-carriers.

Calibration plots according to the presence of an ICD show an acceptable agreement between predictions and observations with a tendency towards overestimation in non-ICD carriers and underestimation in ICD carriers in the higher risk spectrum (see Supplementary material online, ).

Discussion

In this study, we validated the published ARVC risk calculator in an independent cohort of patients from 29 centres in 6 countries in North America and Europe. Since its publication in 2019, the risk calculator had a significant uptake in clinical practice. Ensuring its reproducibility and accuracy in an independent patient population is crucial to ensure both usefulness and safety.

The main findings are as follows:

(1) Demonstration that the model is accurate in its predictions with an adequate discrimination and calibration in a cohort with a sufficient sample size.[10,25] The performance of the risk calculator was indeed comparable to what was reported initially and its prediction accuracy in this cohort would not be improved by recalibration.[21]

(2) Demonstration that the risk calculator generally outperforms various risk factor approaches recommended in published consensus-based algorithms for ICD use in ARVC.

These findings thus support the clinical use of this risk prediction model as a valuable tool for sustained VA and SCD risk stratification in definite ARVC and, consequently, for guiding decisions about primary prevention ICD indications ().

Comparison of the internal and external validation populations

While based on the same inclusion criteria (i.e. definite diagnosis of ARVC and no prior history of sustained VA at the time of diagnosis), the initial risk calculator included a high proportion of patients treated at highly specialized ARVC referral centres. Thus, a significant concern regarding this population is a possible selection bias due to the preferential referral of patients for adverse disease progression (i.e. recurrent VA referred for ablation and severe heart failure for advanced therapies). This could potentially hamper external validity. The present cohort derived from 29 different centres in 6 countries is thus likely to reflect a more diverse ARVC population. Expectedly, the annual event rate in this validation cohort (4.98%, 95% CI 4.07–6.04) was slightly lower, although non-significantly, than in the derivation population (5.6%, 95% CI 4.7–6.6) during a similar follow-up period [5.02 (2.05–7.90) years in the validation versus 4.83 (2.44–9.33) years in the derivation cohort]. This reflects the overall high risk of VA events in definite ARVC patients such as those included in this study which is consistent with prior literature and often preceding structural changes.[4,19,25-28]

Some differences between the two cohorts (shown in Supplementary material online, ) might have limited the potential discrepancy in event rates, such as a higher proportion of probands (64.8% vs. 49.8%, P < 0.001) and males (54.8% vs. 44.7%, P = 0.002) in the current cohort. Conversely, patients in the present cohort were slightly older (43.1 vs. 38.2 years of age, P < 0.001), had less recent cardiac syncope and NSVT. The proportion of patients with decreased LVEF (<50%) was also higher in this cohort (17.7 vs. 12.7, P = 0.002). Although individuals in the current population were more likely to receive anti-arrhythmic drugs (P < 0.001) and β-blockers (48.0 vs. 37.9, P = 0.001), the proportion of ICD carriers at baseline was similar (41.1 vs. 40.8 P = 0.98). Finally, while still representing the predominant genotype, the proportion of patients with PKP2 causal variants was lower than in the derivation cohort (39.4% vs. 51.1% of tested patients) factoring that the current cohort has a lower proportion of patients with known genetic information. This predominance of PKP2 genotype is consistent with prior literature including patients with definite ARVC diagnosis.[29] The proportion of patients with DSP causal variants was also higher (8.9% vs. 4.4%) than in the derivation cohort.

Model performance

The current validation cohort included 429 patients, of whom 103 had events. This met the minimally recommended sample size of 100 patients with and 100 patients without events to attain sufficient power for external validation.[30] The initial study and internal validation using bootstrapping yielded an optimism corrected C-statistic of 0.77 (95% CI 0.73–0.81) and a calibration slope of 0.93 (95% CI 0.92–0.95). In the current study, we obtained comparable results with a slightly lower C-statistic of 0.70 (95% CI 0.65–0.75) showing acceptable discrimination and a calibration slope of 1.01 (95% CI 0.99–1.03) demonstrating almost perfect agreement between predictions and observations for sustained VA. As illustrated in the calibration plot, this concordance between observations and predictions was consistent across the risk spectrum (). Calibration in subgroups based on geographical origin, pedigree position, and genotype did not reveal major discrepancies although the study was not adequately powered to arrive at definitive conclusions in these subgroups.

The results of the current study are consistent with five small studies which have addressed the external validation of the ARVC risk calculator since its publication. The risk calculator was shown to perform well in patients with a definite diagnosis of ARVC[5,6,8] and regardless of their exercise status.[7] The validation study by Baudinaud et al.,[6] on a cohort of 115 patients, only 15 with VA events, of whom only one had an ICD at baseline, reported a C-statistic of 0.84 (CI 0.74–0.93) while reporting an overestimation of the risk in lower risk patients.

The model generally showed a superior net clinical benefit when compared to a risk factor approach as recommended in the three published consensus documents.[2,18,22] The model was similarly shown to outperform the ITFC and HRS consensus in two separate cohorts.[5,6] These studies, however, suggested highly different thresholds for ICD implantation (10% and 37%), assuming an equal weight to unprotected VA and unnecessary ICDs. We did not present such an analysis as we do not propose that these adverse events are equivalent and rather preferred the use of the weighted analysis along with the graphical presentation of the clinical implications of the different threshold. The question of the threshold for ICD implantation is a legitimate concern when using the risk calculator. Establishing a single perfect threshold is a delicate undertaking as every cut-off point comes with a trade-off between unnecessary ICDs with their potential complications versus the potential for unprotected SCA. The relative weight of these opposing undesirable events varies significantly from one individual to another. In the individualized decision-making process; however, a few points should be considered when reflecting on the threshold for ICD use. First, when tempted to use a similar threshold as suggested by the guidelines for the hypertrophic cardiomyopathy (HCM) risk calculator (i.e. ≥6% within 5 years),[31,32] the breakdown of the type of events is relevant. In ARVC cohorts, including the current study and in the derivation cohort, most events were either ICD treated events or sustained VA, while most events in the cohort leading to the HCM risk calculator cohort were SCD or SCA.[33] Although most clinicians agree that sustained or ICD treated VAs represent significant events, supported by guidelines,[2,34] the exact number of treated VA events corresponding to a potential SCD is unknown in ARVC. Another important aspect to consider is that none of these studies are prospective evaluations of the role of ICDs in SCD prevention. Such an undertaking would not be feasible in contemporary high-risk ARVC populations. However, from such prior studies in the general cardiomyopathy population the one which established a benefit for primary prevention ICDs with the lowest annual risk of mortality, SCD-HeFT, had an annual risk of SCD of 3.5%.[35] Finally, the cost of ICDs is rarely a significant determinant nowadays in countries where ICDs can be considered in primary prevention.[36] Factoring the low number of ICDs needed to treat one VA event in ARVC, decreases in the cost of devices, the lifespan of modern ICDs reaching 10 years, and the potential number of quality-adjusted live years (QALY) saved in this young, usually otherwise healthy population (only five individuals had non-arrhythmic death during follow-up in this cohort), the common, although debated thresholds for a QALY between 50 000 and 100 000 USD[37] remains far of reach. Conversely, the rate of short- and long-term complications of ICDs remain significant in ARVC patients (annual rate of complications of 4.2% and of inappropriate shocks of 3.9%),[38] and although subcutaneous-ICDs have become an appealing alternative, there is no evidence of a lesser risk.[39,40]

Thus, in light of these different considerations, we do believe that the best use of the risk calculator is as a shared decision making tool balancing the opposing risks of SCD and ICD use. It appears reasonable that the predicted 5-year risk threshold for recommending an ICD would range from 5% to 25%, depending on the patient’s values and preferences, and the clinician’s judgement. We acknowledge that the threshold may change in the future with advances in non-invasive treatments and innovations in ICD technology which may lower risks associated with devices.

Future improvements in the model

While the model demonstrated a better performance compared to other published decision algorithms, it remains imperfect as illustrated by a C-statistic of 0.70. While it is unlikely that any risk stratification tool for SCD could predict the totality of these events, different elements could potentially improve prediction in the future. The addition of more refined parameters indicating left ventricular involvement, including late gadolinium enhancement were recently suggested.[9] Genotype may also improve SCD risk prediction as recently proposed for patients with phospholamban associated disease.[41] Finally, additional invasive parameters such as programmed ventricular stimulation[42,43] might add additional accuracy in intermediate-risk cases. Moreover, the model is based on prediction of risk from the time of diagnosis of ARVC; a time-updated model for repeated risk prediction may have practical clinical utility.

Limitations

In this study, the majority of sustained VA outcomes are ICD treated events. While this fact is not possible to overcome in most modern ARVC populations and while most would agree that these still represent significant events, they do not directly represent the underlying risk of SCD. However, this is a limitation shared with most of previous studies in this field, including most of those used to elaborate prior consensus-based risk stratification algorithms. While underpowered for events, calibration plots by ICD carrier status show acceptable correlation between predictions and observations. This reflects both that ICDs are implanted in patients believed to be at higher risk (selection bias), but also increase the detection of some arrhythmia that might have gone undetected otherwise (information bias) (see Supplementary material online, ). While family members are well represented in the derivation cohort (50.2%), they are less prevalent in the current cohort (35.2%), and contribute to a lower proportion of events (21.1%). The calibration plot in this specific subgroup, although underpowered, suggests possible overestimation in the lower risk patients which should be taken in consideration when using the model.

Missing data also represent a limitation of this retrospective cohort. Although a complete case analysis reassuringly demonstrates similar results with regard to performance, missing data could influence the relative benefit of the model over consensus-based methods.

Finally, this validation only applies to patients who were well represented in the derivation and validation cohorts. The model should thus not be used in patients who do not meet definite ARVC diagnosis as per 2010 TFC such as those with left dominant forms and in patients with rare malignant genotypes such as TMEM43-p.S358L, of which only 10 patients were included in this cohort.

Conclusion

In this external validation study, we demonstrated that the published ARVC risk prediction model not only provides accurate prognostic information in patients with ARVC without a prior history of sustained VA at diagnosis, but also performs generally better than other published decision algorithms. These findings support its clinical use as a valuable tool for risk stratification enabling consistent and effective shared decision making for ICD implantation.

Supplementary material

Supplementary material is available at European Heart Journal online.

Click here for additional data file.