Tuning and external validation of an adult congenital heart disease risk prediction model.

AIMS: Adequate risk prediction can optimize the clinical management in adult congenital heart disease (ACHD). We aimed to update and subsequently validate a previously developed ACHD risk prediction model. METHODS AND
RESULTS: A prediction model was developed in a prospective cohort study including 602 moderately or severely complex ACHD patients, enrolled as outpatients at a tertiary centre in the Netherlands (2011-2013). Multivariable Cox regression was used to develop a model for predicting the 1-year risks of death, heart failure (HF), or arrhythmia (primary endpoint). The Boston ACHD Biobank study, a prospectively enrolled cohort (n = 749) of outpatients who visited a referral centre in Boston (2012-2017), was used for external validation. The primary endpoint occurred in 153 (26%) and 191 (28%) patients in the derivation and validation cohorts over median follow-up of 5.6 and 2.3 years, respectively. The final model included 5 out of 14 pre-specified predictors with the following hazard ratios; New York Heart Association class ≥II: 1.92 [95% confidence interval (CI) 1.28-2.90], cardiac medication 2.52 (95% CI 1.72-3.69), ≥1 reintervention after initial repair: 1.56 (95% CI 1.09-2.22), body mass index: 1.04 (95% CI 1.01-1.07), log2 N-terminal pro B-type natriuretic peptide (pmol/L): 1.48 (95% CI 1.32-1.65). At external validation, the model showed good discrimination (C-statistic 0.79, 95% CI 0.74-0.83) and excellent calibration (calibration-in-the-large = -0.002; calibration slope = 0.99).
CONCLUSION: These data support the validity and applicability of a parsimonious ACHD risk model based on five readily available clinical variables to accurately predict the 1-year risk of death, HF, or arrhythmia. This risk tool may help guide appropriate care for moderately or severely complex ACHD.

Introduction

Improvement of surgical techniques and medical treatments over the last 70 years has improved the survival of children with a congenital heart defect (CHD). Consequently, the population of adults with congenital heart disease (ACHD) is growing, with associated high healthcare utilization. In this context, there is increased focus on optimizing the clinical management of these patients.

In addition to improvement in medical techniques over the past decades, advancements in technology, and subsequently the development of software programmes, have enabled big data analysis and advanced statistical modelling. One way this has been applied in the current medical era is the development of risk prediction models, which estimate an individual’s risk of a certain outcome given a set of risk factors. This information can help clinicians to inform patients about their prognosis and aids clinical decision-making.

In 2018, our group developed an ACHD risk prediction model, incorporating clinical characteristics including echocardiographic and blood biomarkers that estimated the 4-year risk probability of death, heart failure (HF), or arrhythmia. However, validation of a prediction model is needed to establish its generalizability, and its usefulness in clinical practice. Initially, no prospective data was available to validate the prediction model and retrospectively collected clinical data from a Czech cohort was used for external validation. Moreover, a model providing estimations on a short time horizon may be considered more clinically relevant, as decision-making (e.g. timing clinical follow-up) tends to occur on a time scale of 1–2 years rather than 4 years. In 2012, the Boston ACHD BioBank was established; this large prospective cohort study enrolls ACHD of any type seen at a large referral centre.

To further optimize the clinical usefulness of the previously derived prediction model, we used data from the initial cohort to tune our original ACHD risk prediction model towards 1-year risk estimations of death, HF or arrhythmia, and subsequently externally validated the newly developed model in prospective data from the Boston ACHD BioBank.

Methods

Derivation cohort

A risk prediction model was developed in 2018, using data from a prospective observational cohort study consisting of 602 adults with moderately or severely complex CHD from a tertiary centre in the Netherlands. For this cohort study, consecutive adults were enrolled during routine visits to the outpatient clinic of the Erasmus Medical Centre between 2011 and 2013. Patients with mild CHD (isolated atrial or ventricular septal defect), creatinine >200 μmol, age <18 years-old, or who were currently pregnant were excluded. At the baseline visit, patients underwent a physical examination, electrocardiography, transthoracic echocardiography, and venous blood sampling. Moderately and severely complex ACHD was defined in accordance with the 32nd Bethesda conference classification.

Patients were prospectively followed for the occurrence of adverse cardiovascular events and had yearly scheduled visits to the outpatient clinic for the first four subsequent years after study enrolment. Events were adjudicated until the 1 January 2018 by two researchers, who were blinded to patient characteristics. Written informed consent was obtained from all patients and the study was performed according to the principles outlined in the Declaration of Helsinki. The study was approved by the medical ethics committee of the Erasmus MC. Details have been published previously. This study was conducted in accordance with the TRIPOD statement.

External validation cohort

External validation of the model used data from the Boston ACHD BioBank study, a prospective observational cohort study enrolling patients ≥18 years-old with CHD during an outpatient visit to a tertiary referral centre (The Boston Adult Congenital Heart Service at Boston Children’s and Brigham and Women’s Hospitals). Details of the implementation and design of this study have been published. The current cohort includes patients enrolled between 2012 and 2017, with an N-terminal pro B-type natriuretic peptide (NT-proBNP) measurement from the baseline visit (n = 921). Only patients with moderately or severely complex ACHD were included in the current study, resulting in a total of 749 patients. The study was approved by Boston Children’s Hospital’s Institutional Review Board with reliance on this board by Partners HealthCare/Brigham and Women’s Hospital, and written informed consent was obtained from all participants or their legally authorized representative.

Model development and modifications

The primary endpoint was a composite of all-cause mortality, HF (associated with hospitalisation or initiation/intensification of cardiac medication), or clinically relevant arrhythmia (symptomatic and recorded, or associated with treatment). The original developed model was based on the 4-year risk of all-cause mortality, HF, or arrhythmia using multivariable logistic regression. We modified the logistic regression model to a Cox proportional hazards regression model to enable 1-year risk predictions.

Model development was performed using the 14 initial candidate predictors that had been selected by an expert panel of senior cardiologists at the Erasmus Medical Centre. Because missing data were limited (1.5%), missing data were imputed using single multivariate imputation by chained equations in R (package mice). Supplementary material online, lists the % of missing data of each variable. Univariable and multivariable Cox regression was used to obtain crude and adjusted predictor effects. Variables were then selected for multivariable analysis based on Akaike’s Information Criteria (P < 0.157) and stepwise backward selection method was used to obtain the final model (P < 0.157). The proportional hazard assumption was assessed through Schoenfeld residual plots and proportional hazard test, and there was no indication that the final model violated this assumption. Non-linear and interaction terms were not considered because of limited statistical power. Bootstrap resampling was used to internally validate the model and the slope of the linear predictor obtained from the bootstrap resampling was used for uniform shrinkage of coefficients to adjust for overfitting and to improve external calibration. The cumulative baseline hazard at 1 year was extracted from the Cox regression model to enable prediction of 1-year risks of death, HF, or arrhythmia.

As a secondary analysis, to assess the predictive ability of the score over a longer time horizon, we also used the cumulative baseline hazard at 2 years to predict 2-year risks of the combined endpoint.

External validation

Predictions were calculated by applying the development model to the validation data. Calibration and discrimination of the final model for the 1-year risk of the composite endpoint was assessed using a calibration plot and the C-statistic, respectively. Calibration-in-the-large was assessed by the difference in the observed mean risk minus the predicted mean risk. The calibration slope was estimated to assess whether the model was overfitted (slope < 1) or underfitted (slope > 1). The same methodology was used to externally validate the model for predicting the 2-year risks of the composite endpoint. When the external performance of the model is adequate, the model will be refitted on a merged dataset consisting of both the derivation and validation data. Statistical analyses were performed using R version 3.6.1 (packages rms, survival, mice).

Results

Characteristics of derivation and validation cohorts

A schematic overview of the study is shown in Figure . Baseline characteristics of both the derivation cohort (n = 602) and the validation cohort (n = 749) are listed in Table . Patients in the validation cohort tended to have characteristics consistent with higher risk. Severely complex CHD was more common in the validation cohort (39%) than in the derivation cohort (29%), and the patients in the validation cohort were also more symptomatic [New York Heart Association (NYHA) functional class > I], were more commonly prescribed cardiac medication, and had a higher prevalence of reintervention after initial repair. Despite these differences, NT-proBNP levels did not differ substantially between cohorts. Age distributions of both studies were different, with Boston having a slightly older study cohort (Supplementary material online, ).

Schematic overview of the development and validation process of the ACHD risk prediction model. The ACHD risk prediction model provides estimates of the 1-year risk of death, heart failure, or arrhythmia for moderately and severely complex ACHD patients based on five clinical variables. Online web application of the prediction model is accessible at https://achdwebcalculator-updated.shinyapps.io/achdshinyweb/, or scan the QR code. ACHD, adult congenital heart disease.

Follow-up data on endpoints were available for 597 out of 602 patients (99.2%) in the derivation cohort and for 692 out of the 749 patients (92.4%) in the validation cohort. No significant differences were observed with regard to the age, sex, body mass index (BMI), heart rate, complexity of CHD, or NT-proBNP between those patients with and without follow-up in the validation cohort. However, those without follow-up were less often prescribed cardiac medication (42 vs. 59%, P = 0.017) (Supplementary material online, ). Median follow-up in the derivation cohort was 5.6 (interquartile range 4.6–6.1) years, by which time the primary endpoint had occurred in 153 patients (25.6%). In the validation cohort, the median follow-up was 2.3 (1.2–3.6) years and 191 patients (27.6%) reached the primary endpoint. After 1 year of follow-up, the endpoint had occurred in 52 patients (8.7%) in the derivation cohort and in 91 patients (13.1%) in the validation cohort. Table  outlines the specific components of the primary endpoint in each cohort.

Model development

Ten of the 14 pre-specified candidate predictors were associated with the probability of developing the primary endpoint in univariable Cox analysis; these variables were considered for inclusion in multivariable analysis (Table ). Multivariable analysis identified five independent predictors of the primary endpoint; NYHA class [NYHA I (0) vs. NYHA II/III (1)], cardiac medication use [no (0) vs. yes (1) including: angiotensin-converting enzyme inhibitor, angiotensin receptor blocker, beta-blocker, diuretics including loop/thiazide/potassium sparing, calcium channel blocker, or anti-arrhythmic drug]; ≥1 cardiovascular reintervention [no (0) vs. yes (1)] BMI (kg/m2); and log2 NT-proBNP (pmol/L). Age and congenital diagnosis (moderately vs. severely complex) were included as predictors in the originally developed logistic regression prediction model but were not independent predictors in the re-developed Cox model. After backward selection, the model included: NYHA class, cardiac medication use, ≥1 reintervention, BMI, and log2 NT-proBNP (Table ). The optimism-adjusted C-statistic for predictions at 1 year was 0.81. After application of the bootstrap slope for shrinkage (0.914), the hazard ratios of the final predictors were: NYHA class ≥II 1.82, cardiac medication 2.33, ≥1 reintervention 1.50, BMI 1.03/kg/m2, and log2 NT-proBNP 1.43/pmol/L. The baseline hazard at 1-year was estimated at 1.06.

The C-statistic of the derived model in the external validation dataset was 0.79 [95% confidence interval (CI) 0.74–0.83]. Calibration-in-the-large, calculated as the mean predicted probability (=0.135) minus the mean observed probability (=0.133), was excellent (−0.002, ideally equal to zero); this indicates that the predicted probabilities neither systematically overestimated or underestimated the true probability of an event. The calibration slope was 0.99 (95% CI 0.85–1.14), suggesting minimal model overfitting (ideally 1) (Figure ). Patients categorized in the highest sextile of predicted risk had an observed risk at 1-year of 41.2%, compared with 1.7% in the lowest sextile. The distributions of the predicted probabilities for the derivation and validation cohort are shown in Figure ; 1-year predicted probabilities were higher in the validation cohort compared to the derivation cohort, in agreement with the higher 1-year observed risk. The endpoint-free survival according to sextile of the 1-year predicted risk for both the derivation and the validation cohort is shown in Supplementary material online, . The excellent calibration allowed us to refit the model based on data from both the derivation and validation cohorts.

Calibration plot of the external validation of the prediction model by the plotted probabilities of the 1-year risk of death, heart failure, or arrhythmia. Diagonal line shows ideal calibration where predicted probability is equal to the observed probability.

Density plot showing the distribution of the 1-year predicted probabilities for the composite outcome of death, heart failure, or arrhythmia in the development cohort and validation cohort. Mean and median (25th–75th percentile) predicted probability for each cohort is given. The predicted probabilities of adverse outcome are higher in the validation cohort, which aligns with the higher observed probability.

Hazard ratios of the model variables in the merged dataset were very similar to those seen in the derivation cohort: NYHA class ≥II = 1.91 (95% CI 1.50–2.43), cardiac medication use = 2.53 (95% CI 1.92–3.35), ≥1 reintervention = 1.55 (95% CI 1.20–2.00), BMI = 1.03 (95% CI 1.01–1.05) kg/m2; and log2 NT-proBNP = 1.44 (95% CI 1.35–1.54) pmol/L. The risk calculator is available online at https://achdwebcalculator-updated.shinyapps.io/achdshinyweb/. The formula for calculating 1-year risk predictions is provided in Supplementary material online, , to allow validation of the model by other investigators. The 1-year risk predictions based on the combined data likewise discriminated well which patients would have endpoint-free survival over a 6-year horizon (Figure ). The observed risk for patients in the highest sextile of predicted risk was 37.7% vs. 0.5%, for patients in the lowest sextile (Supplementary material online, ).

Cumulative endpoint-free survival in 1289 adults with congenital heart disease according to sextile of the ACHD prediction model 1-year predicted risk of the composite endpoint. The endpoint was a composite of all-cause mortality, heart failure (associated with hospitalization or initiation/intensification of heart failure medication) or arrhythmia (symptomatic and recorded, or associated with treatment). ACHD, adult congenital heart disease.

Model performance for the 2-year predicted risks of the endpoint was similar to the 1-year fitted risk prediction model; calibration slope = 0.992, calibration-in-the-large = 0.004, C-index = 0.77 (Supplementary material online, ).

Discussion

This study leveraged data from two large prospectively enrolled cohort studies of adults with CHD to re-develop and subsequently validate a parsimonious ACHD risk prediction model. The final model consisted of five predictors: NYHA class, cardiac medication use, reintervention, BMI, and NT-proBNP. The external validation in 749 patients from the Boston ACHD BioBank study suggests the model accurately predicts the 1-year risk of arrhythmia, HF, or death, and discriminates well between patients who suffer a clinically important cardiovascular outcome in different clinical contexts. The excellent calibration of this model in a large, independent, prospective cohort study, strengthens its validity and generalizability to a diversity of moderately and severely complex ACHD diagnoses.

Comparison with previous model development and validation

The broad spectrum of CHD diagnoses with variable disease severity make it challenging to apply a single prediction model to all adults with CHD. However, in the current study, moderately vs. severely complex ACHD was not an independent predictor for the occurrence of HF, arrhythmia or all-cause mortality. There may be several explanations; first, the complexity of the CHD may be partially captured by other variables included in the model, minimizing additive information. Severely complex ACHD patients may tend to use cardiac medication more frequently, for example, and are more likely to have NYHA II or III symptoms, and to undergo reinterventions. Second, the distinction between moderately and severely complex diagnoses could be insufficient to account for heterogeneity across congenital diagnoses as disease severity can vary greatly within moderately and severely complex ACHD patients. The American College of Cardiology/American Heart Association recently published ACHD guidelines which propose a new ACHD classification, comprised of both CHD anatomic class and variables reflecting the patient’s ‘physiological stage’. This was done to create better risk stratification in ACHD and better guide follow-up; recently this proposed classification has been assessed by Ombelet et al. which showed that addition of the physiological stage of patients to the anatomic class improves long-term prediction of cardiac mortality. The results of our study support this approach conceptually, and the need for differentiation based on more characteristics than the severity of the CHD itself. Nevertheless, these results do not undermine the need for diagnosis-specific predictors, as each ACHD diagnosis may have idiosyncratic predictors for outcomes (e.g. hypertension in coarctation of the aorta). Another study by Ombelet et al. showed that NYHA class and other functional indices can predict 15-year mortality in ACHD. Our study shows that cardiac medication, prior reinterventions, BMI, and NT-proBNP provide additional prognostic information on top of NYHA class for a combined endpoint. While NYHA class is a key clinical variable strongly associated with prognosis, combining functional indices with other clinical variables and more nuanced measurements such as NT-proBNP, seems to improve risk prediction. After all NYHA functional class is limited to four possible risk strata and in practice 3; almost all patients at a given time will be classified as NYHA class 1, 2, or 3.

Cohen et al. developed a prediction model to predict the 1-year risk of ACHD HF hospitalization based on a large administrative database in Quebec. That cohort had substantially lower risk: 0.72% 1-year risk of HF hospitalization, as compared with 3.5% in the current cohort. The presence of lifetime comorbidities among which diabetes mellitus, coronary artery disease, and chronic kidney disease, increased the 1-year risk of HF hospitalization. In the current study, comorbidities were not considered as predictors because of the relatively low prevalence; but presumably the presence of these diagnoses are also associated with worse prognosis in this sort of patient.

Age was not included in the final model in the current study. One may argue that, specifically for mortality, age should always be included as a predictor, since it is related to survival prospects. In a meta-analysis of HF from acquired causes, age was an independent predictor for mortality. Age may have lacked predictive value in our study because of the relatively narrow age range, the inclusion of younger patients who were sicker based on timing of clinical visits, or due to the use of a composite endpoint. However, this finding is not unique to our study. Yap et al., for example, found that age was not a predictor of mortality in a study of 378 patients with ACHD and atrial arrhythmia, even in univariable analysis. In addition, a systematic review assessing predictors for HF in ACHD, reported that age was a significant predictor in only 25% of the studies. Age may therefore be less relevant for prognosis in ACHD compared to variables more directly reflecting clinical condition, such as NYHA functional class or cardiac medication use.

The higher predicted risks in the validation cohort parallel the higher observed risks, indicate that the predictive value of the variables behave similarly in cohorts with different overall disease severity. The C-index was 0.79, indicating the model discriminates well between patients who do and do not experience the composite outcome. Further improvement would be desirable, and additional predictors, both general and diagnosis-specific, not considered in the initial derivation, should be explored to enhance discrimination of high- and low-risk patients.

External validation and generalizability

Both cohort studies are likely realistically representative of the overall population of ambulatory, moderately to severely complex ACHD patients engaged in specialist care, since few exclusion criteria were applied in either study. This is important for the generalizability of a model. The observed and the predicted probabilities in the Boston cohort were considerably higher compared to the Rotterdam cohort. A likely explanation is referral selection bias of more complex and ill patients to Boston Children’s Hospital/Brigham and Women’s Hospital, as suggested by the higher proportion of patients on cardiac medication and NYHA functional class >1. Low-risk patients may be less likely to have scheduled follow-up over a short-to-medium time period compared to high-risk patients, leading to a relatively sicker cohort of ACHD patients; there was no defined follow-up period, in distinction to the annual follow-up for all patients in Rotterdam. The vastly different healthcare systems may in part explain differences in follow-up including regionalization of care in the Netherlands, as might the more expansive geography of referral in the USA and Boston.

Generalizability of this model should be evaluated in low- and middle-income countries or in other situations where ACHD care and therapeutic strategies may differ. Moreover, this model depends on measurement of NT-proBNP, restricting the use of this model to medical centres able to perform this test.

Clinical usefulness

The ACHD risk calculator provides reliable estimations of the 1-year risk of death, HF, or arrhythmia and can provide context to help guide the follow-up strategy of ACHD patients. For instance, follow-up timing may be adjusted based on the anticipated medium-term risk (e.g. if 2-year risk is <5%, it may be reasonable to defer the next regular annual visit unless there is a change in symptoms). Ultimately, the usefulness of the tool depends on being able to identify interventions or other diagnostic tests that will improve a given patient’s care, whether in terms of avoiding clinical events or enhancing the quality of life. This might not only take the form of increased surveillance or intervention; avoiding unnecessary clinical visits for patients at very low risk may also substantially enhance patient satisfaction and quality of life.

The Cox model assesses relative risk, and therefore the predicted probabilities depend on the population in which the model is developed. Therefore, absolute risk estimates should be interpreted cautiously when applied to other clinical contexts. Nevertheless, this study shows that the risk prediction model can accurately identify high- and low-risk patients and that the predicted risks were calibrated well in an external cohort. The question remains whether the model improves clinical decision-making. This can be assessed using a decision curve analysis with specified cut-offs for high- vs. low-risk and their treatment indication. A randomized controlled trial of physicians using the risk model to aid clinical decision-making vs. physicians not using the model would be needed to objectively assess its clinical effectiveness and cost-efficiency.

Limitations

Due to insufficient power, additivity and linearity assumption could not be relaxed by including interaction terms and non-linear terms. This may have hindered the discriminative ability, though it has been shown that these terms do not often have considerable influence on the model performance, as interaction- and non-linear terms are likely to induce model overfitting.

This model was developed using a composite endpoint. While all three components are substantive, ‘hard’ clinical events, their importance is not equivalent and the strongest risk factors may vary between these three events. Furthermore, the diagnostic and therapeutic strategies relevant to a patient at high risk for arrhythmia may differ from a patient at high risk of developing HF.

We tested 14 pre-selected variables as candidate predictors. Other predictors may be strong predictors of outcome but may have been too uncommon to enable adequate power (e.g. severe cyanosis), were not relevant to both cohorts (e.g. health insurance status, race) or were not available in both datasets (e.g. cardiopulmonary exercise test data).

Conclusions

The ACHD risk calculator, based on data from two large prospective cohort studies including a total 1351 moderately to severely complex ACHD patients, provides reliable estimates for the 1-year risk of death, HF or arrhythmia based on five readily available clinical variables; NYHA functional class, cardiac medication use, reinterventions, BMI, and NT-proBNP. The resulting risk prediction tool is suitable for assistance of risk stratification in clinical practice when caring for moderately and severely complex ACHD patients. We encourage other investigators to collaborate on additional validation and improvement of the current risk model or derivation of diagnosis-specific and outcome-specific models to better focus clinical care on specific risks.

Supplementary material

Supplementary material is available at European Heart Journal – Quality of Care and Clinical Outcomes online.

Funding

This work was conducted with support from Harvard Catalyst | The Harvard Clinical and Translational Science Center (National Center for Research Resources and the National Center for Advancing Translational Sciences, NIH Award UL1 TR001102) and financial contributions from Harvard University and its affiliated academic healthcare centres. The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic healthcare centres, or the National Institutes of Health. This work was also supported by an investigator-initiated study research grant from Roche Diagnostics (Indianapolis, IN, USA). The Erasmus Trustfonds provided a travel grant to L.W.G. in support of this study. A.R.O. and M.J.L. are partly supported by the Dunlevie Family Fund.

Conflict of interest: none declared.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author.

Click here for additional data file.

Table 1

Baseline clinical characteristics of included ACHD patients in the derivation cohort (Rotterdam) and the validation cohort (Boston)

	Derivation cohort	Validation cohort	P-value
Clinical characteristics	Rotterdam (n = 602)	Boston (n = 749)
Age (years)	32 (25–41)	36 (27–48)	<0.001
Woman	254 (42)	361 (48)	<0.001
Congenital diagnosis
Moderate	429 (71)	458 (61)	<0.001a
Tetralogy of Fallot or DORV	179 (30)	175 (23)
Aortic coarctation	112 (19)	79 (11)
LV obstructive disease	138 (23)	78 (10)
AVSD	0 (0)	37 (5)
Ebstein anomaly	0 (0)	26 (3)
Other	0 (0)	63 (8)
Complex	173 (29)	291 (39)	<0.001a
TGA—arterial switch	24 (4)	29 (4)
TGA—Mustard or Senning	65 (11)	53 (7)
TGA—congenitally corrected	21 (3)	26 (3)
Fontan	36 (6)	133 (18)
Pulmonary arterial hypertension/Eisenmenger	9 (1)	16 (2)
Rastelli/REV procedure	11 (2)	10 (1)
Univentricular heart, palliated or unoperated	7 (1)	9 (1)
Pulmonary atresia with intact ventricular septum	0 (0)	14 (2)
Other	0 (0)	1 (0)
NYHA class			<0.001
Class I	541 (90)	551 (74)
Class II	56 (9)	166 (22)
Class III/IV	5 (1)	28 (4)
Cardiac medication use	212 (35)	433 (58)	<0.001
≥1 reintervention after initial repair	317 (53)	447 (63)	<0.001
Heart rate (beats/min)	74 ± 13	71 ± 13	0.013
Systolic blood pressure (mmHg)	126 ± 16	120 ± 14	<0.001
Oxygen saturation <90%	17 (3)	38 (6)	0.037
Body mass index (kg/m2)	24.7 ± 4.4	26.9 ± 5.6	<0.001
Current tobacco use	56 (10)	36 (5)	<0.001
Sinus rhythm on baseline ECG	521 (87)	—	—
Systemic ventricular function			<0.001
Normal	303 (50)	497 (71)
Mildly impaired	212 (35)	142 (20)
Moderately/severely impaired	87 (15)b	63 (9)
NT-proBNP (pmol/L)	15 (7–33)	16 (7–40)	0.193

Data are presented as mean ± SD, median (25th–75th percentile) for continuous variables (normally and non-normally distributed, respectively), and n (%) for categorical variables.

AVSD, atrioventricular septal defect; DORV, double outlet right ventricle; LV, left ventricular; NT-proBNP, N-terminal pro B-type natriuretic peptide; NYHA, New York Heart Association; REV, Réparation à l'Etage Ventriculaire; TGA, transposition of the great arteries.

P-value for comparison moderate vs. complex ACHD.

n = 18 patients severely impaired systemic ventricular function.

Table 2

Proportion of patients experiencing each of the individual components of the composite endpoint in the derivation and validation cohorts

Variable	Derivation cohort (n = 597)	Validation cohort (n = 692)
Follow-up (years)	5.6 (4.6–6.1)	2.3 (1.2–3.6)
Primary composite endpoint
During entire follow-up duration	153 (25.6)	191 (27.6)
Reached after 1 year of follow-up	52 (8.7)	91 (13.1)
Death	25 (4)	44 (6)
End-stage heart failure	10 (2)	9 (1)
Sudden death/cardiac arrest	10 (2)	6 (1)
Other or unknown	5 (1)	29 (4)
Heart failure	59 (10)	120 (17)
Hospital admission	25 (4)	37 (5)
Initiation or intensification in diuretics	34 (6)	83 (12)
Arrhythmia	128 (21)	109 (16)
Ventricular tachycardia/fibrillation	31 (5)	10 (1)
Supraventricular tachycardia	84 (14)	92 (13)
Other	13 (2)	7 (1)

n (%) are shown for individual event components. Continuous variables are presented as median (25th–75th percentile). Separate event components of the primary endpoint are shown (i.e. patients were not censored at the time of another endpoint event than the endpoint of interest). For heart failure and arrhythmia, only the earliest occurrence is listed (e.g. a patient who had intensification of diuretics and was later hospitalized for heart failure).

Table 3

Cox regression analysis for the 14 pre-specified candidate predictors in the derivation cohort

Variable	Univariable HR (95% CI)	P-value	Multivariable HR (95% CI)	P-value	Final model HR (95% CI)	P-value
Age, per year	1.05 (1.04–1.07)	<0.001	1.01 (0.99–1.02)	0.360
Sex, male vs. female	0.80 (0.58–1.09)	0.159	—	—
Congenital diagnosis complexity, severely vs. moderately	2.26 (1.64–3.11)	<0.001	1.20 (0.78–1.84)	0.407
NYHA class, II–III vs. I	5.62 (3.95–8.00)	<0.001	1.90 (1.22–2.95)	0.004	1.92 (1.28–2.90)	0.002
Cardiac medication use, yes vs. no	5.14 (3.66–7.21)	<0.001	2.41 (1.60–3.62)	<0.001	2.52 (1.72–3.69)	<0.001
≥1 reintervention after corrective repair, yes vs. no	2.29 (1.62–3.23)	<0.001	1.64 (1.13–2.40)	0.010	1.56 (1.09–2.22)	0.015
BMI, per kg/m2	1.05 (1.02–1.09)	0.003	1.03 (1.00–1.07)	0.044	1.04 (1.01–1.07)	0.020
Heart rate, per beat/min	1.00 (0.99–1.01)	0.953	—	—
Current tobacco use, yes vs. no	0.82 (0.48–1.39)	0.460	—	—
Oxygen saturation <90% vs. ≥90%	2.59 (1.32–5.07)	0.006	0.76 (0.37–1.56)	0.449
Loss of sinus vs. sinus rhythm at baseline ECG	3.24 (2.28–4.61)	<0.001	0.90 (0.59–1.38)	0.633
Systemic ventricular function, 0–3a	1.85 (1.57–2.19)	<0.001	0.99 (0.81–1.22)	0.952
Severe valvular dysfunction, yes vs. no	1.19 (0.78–1.83)	0.427	—	—
NT-proBNP, per log2 pmol/L	1.80 (1.64–1.98)	<0.001	1.44 (1.27–1.64)	<0.001	1.48 (1.32–1.65)	<0.001

Visually graded based on echocardiography as normal (0), mildly (1), moderately (2), and severely (3) impaired systemic ventricular function (analysed as continuous variable 0–3)

BMI, body mass index; ECG, electrocardiogram; NT-proBNP, N-terminal pro B-type natriuretic peptide; NYHA, New York Heart Association.