Predicting long-term freedom from atrial fibrillation after catheter ablation by a machine learning algorithm: Validation of the CAAP-AF score.

BACKGROUND: Preprocedural clinical predictors of the successful maintenance of sinus rhythm may contribute to optimal treatment strategies for atrial fibrillation (AF). The CAAP-AF score, a novel simple tool scored as 0-13 points (including six independent variables) has been proposed to predict long-term freedom from AF after catheter ablation. To clarify its reproducibility, we examined the CAAP-AF score's predictive performance and then created subgroups to best predict AF recurrence by using a machine learning algorithm.
METHODS: We studied 583 consecutive patients who underwent initial AF catheter ablation at our institute (median CAAP-AF score, 5; age, 66 ± 10 years old; female, 28.3%; coronary artery disease, 10.8%; left atrial diameter, 39.9 ± 6.6 mm; number of antiarrhythmic drugs failed, 0.4 ± 0.6; nonparoxysmal AF, 45.3%). All were systematically followed up with an endpoint of atrial tachyarrhythmia recurrence after the last ablation procedure.
RESULTS: During the 1.8 ± 1.2-year follow-up, 157 patients had atrial tachyarrhythmia recurrence. Repeated procedures were performed (n = 115). Arrhythmia recurrence after the last session occurred in 69 patients. We created Kaplan-Meier curves for freedom from AF after final AF ablation for ranges of CAAP-AF scores; these confirmed the original study results. The machine learning using Classification and Regression Trees divided the patients into three categories by the risk score: low (score ≤5), intermediate (score 6-8), and high (score ≥9).
CONCLUSIONS: The CAAP-AF score was useful to stratify the atrial tachyarrhythmia recurrence risk in AF patients undergoing catheter ablation into three categories. The score should be considered when deciding whether to perform AF ablation in clinical practice.

INTRODUCTION

Catheter ablation of atrial fibrillation (AF) is effective for restoring and maintaining sinus rhythm in patients with paroxysmal AF (PAF) or persistent AF (PEF), and probably in patients with long‐standing persistent AF (LSPEF).1 However, sinus rhythm without severely symptomatic recurrences of AF occurs in up to 70% of patients with PAF, and in approx. 50% of patients with PEF.2, 3, 4 The identification of preprocedural clinical predictors of success in maintaining sinus rhythm could thus help to select the optimal treatment strategy for patients with AF.

Various independent clinical predictors related to AF recurrence after catheter ablation have been demonstrated. For example, the size of the left atrium (LA),5, 6, 7 the type of AF,5, 6, 8 gender,9 and patient age at the timing of catheter ablation8, 10 are related to the recurrence of AF after catheter ablation. The CAAP‐AF score, a novel simple score consisting of six independent variables, has been proposed to predict long‐term freedom from AF after ablation, based on data obtained at a single center.11 “CAAP‐AF” stands for the presence/absence of Coronary artery disease, the left Atrial diameter, Age, the presence of Persistent or long‐standing AF, the number of Antiarrhythmic drugs failed, and Female gender. In the Kaplan‐Meier curves of freedom from AF after final ablation classified by the CAAP‐AF score, the AF recurrence rate tended to rise as the score increased.11 The CAAP‐AF score may provide a realistic expectation of freedom from AF following ablation for individual patients, but the performance of the score has not been well validated. In addition, since the CAAP‐AF score ranges from 0 to 13, it seems complicated to directly apply the score to the risk stratification of patients. We hypothesized that a classification of patients into several subgroups at similar risk could better apply the CAAP‐AF score in clinical practice.

The present study had two aims. We first examined the predictive performance of the CAAP‐AF score in order to clarify its reproducibility. We then created subgroups to best predict AF recurrence, by using a machine learning algorithm.

PATIENTS AND METHODS

This study was conducted at Ogaki Municipal Hospital, Department of Cardiology, Ogaki, Japan. The study complied with the Declaration of Helsinki. The database was built prospectively, and the patients' records were reviewed with the approval of the institutional review board at Ogaki Municipal Hospital.

Patient population

A total of 583 consecutive patients who underwent initial AF ablation at our institute were registered from March 2012 to August 2016. Each patient's type of AF was determined based on the 2017 HRS/EHRA/ECAS/APHRS/SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation.1 In the present study, PAF was defined as episodes of AF lasting for <7 days that subsequently reverted to sinus rhythm. PEF was defined as AF episodes lasting ≥7 days, including episodes that are terminated by cardioversion, either with drugs or by direct current cardioversion, after ≥7 days. LSPEF was defined as AF episodes lasting ≥1 year.

Ablation protocol

Patients were effectively anticoagulated for >3 weeks, and a transesophagus echocardiography was performed to exclude any LA thrombi prior to the catheter ablation. Before the procedure, all antiarrhythmic drugs were discontinued for ≥5 half‐lives (except for amiodarone). The ablation procedure was performed under local anesthesia with mild conscious sedation. Radiofrequency (RF) or cryoballoon (CB) ablation was performed in all patients under the guidance of a three‐dimensional mapping system (a NavX system, St. Jude Medical; or a CARTO system, Biosense Webster).

An activated clotting time was maintained between 300 and 350 s. In the RF group, the patients underwent circumferential pulmonary vein (PV) isolation with point‐by‐point applications using an irrigation tip catheter to create the contiguous lesions. In the Cryoablation group, with the use of a second‐generation CB ablation catheter (Arctic Front Advanced Cardiac Cryoablation Catheter: Medtronic), all patients underwent individual PV isolation. If isolation was not achieved by two or more CB ablations, touch‐up ablation was added with an RF or cryoablation catheter. The success of PV isolation was defined as a bidirectional conduction block between the LA and PVs. At the discretion of the operators, additional procedures (eg, posterior wall isolation, LA roof or anterior linear ablation, superior vena cava isolation, non‐PV foci ablation, or cavotricuspid isthmus ablation) were added.

Follow‐up

Patients underwent a regular follow‐up at our outpatient clinic after the ablation procedures at 1, 3, 6, and 12 months, and then annually. At each visit, a medical history was obtained, a physical examination was performed, and 12‐lead electrocardiography (ECG) and 24‐hour Holter monitor recording were obtained. If patients had symptoms, ambulatory electrocardiographic monitoring using a portable electrocardiograph (HCG‐801; Omron Healthcare) was performed on the patients to correlate the findings with the symptoms. Three months after the procedure, discontinuation of antiarrhythmic drugs was encouraged. Patients with initial failures were encouraged to undergo repeat ablation after the 3‐month blanking period. The missing follow‐up data were obtained by contacting the patient or the patient's attending physician.

Study endpoint

The study endpoint was the recurrence of any documented episode of atrial tachyarrhythmia lasting >30 s with or without the use of antiarrhythmic drugs. We defined 90 days after the patient's last AF ablation procedure as the blanking period. However, if repeated ablation was performed within 90 days after the last session, the patient was considered to have recurrent arrhythmia at day 91.

Statistical analysis

Continuous variables are presented as means with standard deviations for normally distributed data. In non‐normally distributed data, they are expressed as medians and interquartile ranges. Discrete variables are given as absolute values or percentages. For the CAAP‐AF scoring, we compared AF‐free survival after final ablation (Kaplan‐Meier curves) with the log‐rank test. We performed a Cox regression analysis to assess the risk of incidence of AF recurrence. The regression analysis results are presented as hazard ratios (HRs) and 95% confidence intervals (95% CIs).

In order to create subgroups to best risk stratify the patients, we divided the patients by using a survival classification and regression tree (CART) model.12 CART is a useful method for estimating suitable cutoff values when predicting the event rate of time‐to‐event data with a continuous variable. We calculated the most significant classification by the machine learning method, and the results are expressed as the regression tree model. In all tests, P < .05 were accepted as significant. Statistical analyses were performed with SPSS software ver. 23 (SPSS) and R ver. 3.4.0: R Core Team (2017) (A language and environment for statistical computing. R Foundation for Statistical Computing; http://www.R-project.org/).

RESULTS

Patients

The characteristics of the 583 patients with AF (418 males [71.7%] and 165 females [28.3%]) are summarized in Table 1. The patients' mean age was 66 ± 10 years old; hypertension was present in 58.8% of the patients, and the mean CHADS2 score was 1.4 ± 1.1. Approximately half of the patients had PAF (54.7%). Table 2 shows the initial ablation procedural characteristics of the patients. An RF ablation was performed in 89.7% of the patients. PV isolation was done in all of the patients, followed by cavotricuspid isthmus ablation (n = 560, 96%) and LA posterior wall isolation (n = 70, 12%).

Table 1

The baseline clinical characteristics of the patients assigned by the risk status

Variable	Total (n = 583)		Low risk (n = 398)	Intermediate risk (n = 170)	High risk (n = 15)	P
Age, y	66 ± 10		65 ± 11	70 ± 7	73 ± 4	<.0001
Gender, female, n (%)	165 (28.3%)		89 (22.4%)	68 (40%)	8 (53.3%)	<.0001
BMI, kg/m2	23.8 ± 3.5		23.6 ± 3.5	24.2 ± 3.5	24.3 ± 4.3	.164
Underlying disease, n (%)
Hypertension	343 (58.8%)		217 (54.5%)	117 (68.8%)	9 (60.0%)	.007
Diabetes mellitus	107 (18.4%)		62 (15.6%)	41 (24.1%)	4 (26.7%)	.039
History of heart failure	118 (20.2%)		49 (12.3%)	61 (35.9%)	8 (53.3%)	<.0001
Systemic embolism or TIA	53 (9.1%)		33 (8.3%)	20 (11.8%)	0 (0%)	.194
Cardiomyopathy (dilated or hypertrophic)	32 (5.5%)		21 (5.3%)	9 (5.3%)	2 (13.3%)	.683
Coronary artery disease	63 (10.8%)		28 (7.0%)	27 (15.9%)	8 (53.3%)	<.0001
CHADS2 score	1.4 ± 1.1		1.2 ± 1.0	1.8 ± 1.2	1.7 ± 0.8	<.0001
CHA2DS2‐VASc score	2.3 ± 1.5		2.0 ± 1.4	3.1 ± 1.4	3.5 ± 1.2	<.0001
BNP, pg/mL [1st‐3rd quartile]	101 [39‐182]		67 [29‐131]	167 [112‐282]	320 [148‐498]	<.0001
eGFR, mL/min/1.73 m2	67.6 ± 18.1		70.6 ± 17.7	61.8 ± 17.5	55.9 ± 14.3	<.0001
No. of drugs failed	0.4 ± 0.6		0.3 ± 0.6	0.4 ± 0.7	0.8 ± 0.6	.0009
LA diameter, mm	39.9 ± 6.6		37.2 ± 5.2	45.0 ± 4.9	52.4 ± 5.9	<.0001
LVEF, %	64.0 ± 9.7		65.7 ± 8.2	60.7 ± 11.4	56.7 ± 11.8	<.0001
AF type, n (%):
PAF	319 (54.7%)	295 (74.1%)		23 (13.5%)	1 (6.7%)	<.0001
PEF	161 (27.6%)	57 (14.3%)		95 (55.9%)	9 (60.0%)
LSPEF	103 (17.7%)	46 (11.6%)		52 (30.6%)	5 (33.3%)
OAC type, n (%)
Warfarin	117 (20%)	70 (17.6%)		40 (23.5%)	7 (46.7%)	.009
DOAC	466 (80%)	328 (82.4%)		130 (76.5%)	8 (53.3%)
Mean number of procedures	1.2 ± 0.5	1.2 ± 0.4		1.2 ± 0.4	1.5 ± 0.8	.12

Data are percentages and absolute numbers or mean ± SD.

Abbreviations: AF, atrial fibrillation; BMI, body mass index; BNP, B‐type natriuretic peptide; DOAC, direct oral anticoagulant; LA, left atrium; LSPEF, long‐standing persistent AF (PEF); LVEF, left ventricular ejection fraction; OAC, oral anticoagulant; PAF, paroxysmal AF; TIA, transient ischemic attack.

Table 2

The index ablation procedural characteristics of the patients

Procedure, n (%)
RF/Cryoballoon	523 (89.7%)/60 (10.3%)
PV isolation	583 (100%)
Posterior wall isolation	70 (12%)
LA roof linear ablation	23 (4%)
LA anterior linear ablation	6 (1%)
SVC isolation	47 (8%)
Non‐PV foci ablation	6 (1%)
CTI linear ablation	560 (96%)

Data are percentages and absolute numbers.

Abbreviations: CTI, cavotricuspid isthmus; LA, left atrium; PV, pulmonary vein; RF, radio frequency; SVC, superior vena cava.

Figure 1 demonstrates the distribution of the patients' CAAP‐AF scores, with the median score of 5.

Figure 1

The patients' CAAP‐AF score values

Ablation outcomes

During the follow‐up period of 1.8 ± 1.2 years, 157 patients had atrial tachyarrhythmia recurrence. Repeated procedures were performed in 115 patients. Arrhythmia recurrence after the last session occurred in 69 patients. In accord with the original paper11, we created Kaplan‐Meier curves for freedom from AF after final AF ablation for the ranges of CAAP‐AF scores, as illustrated in Figure 2A. The patients with higher CAAP‐AF scores were significantly associated with a higher rate of AF recurrence during the follow‐up period (P < .01). In addition, we created a modified version of the Kaplan‐Meier curves by separating a CAAP‐AF score of 8 from a score of 9‐13. The arrhythmia‐free survival curve of the patients with a score of 8 was close to those of the patients with scores of 6 and 7 (Figure 2B). There was a positive correlation between a CAAP‐AF score of 4‐12 and the HR of recurrence with the score 0‐3 as the reference (Table 3). However, the differences were not significant between the patients with the CAAP‐AF scores of 0‐3 and those with the score of 4, or between the patients with scores of 0‐3, and those with the score of 5 (Table 3). There were some cross‐overs among the survival curves during the follow‐up period (Figure 2).

Figure 2

The Kaplan‐Meier survival curves for freedom from atrial tachyarrhythmia after final AF catheter ablation based on the patients' CAAP‐AF scores: A, The patients were divided into six groups based on the CAAP‐AF score following the original paper.11 B, The survival curve of the patients with score 8 is separated from those with score 9‐13

Table 3

Risk of atrial tachyarrhythmia recurrence after last catheter ablation by CAAP‐AF score: original classification

CAAP‐AF score	Patients, n	Patients with recurrence, n	HR	95% CI	P
0‐3	173	13	1	–	–
4	94	9	1.206	0.516‐2.822	.665
5	106	11	1.548	0.693‐3.458	.287
6	73	13	3.066	1.413‐6.651	.005
7	59	9	2.355	1.003‐5.525	.049
8‐12	42	12	4.882	2.208‐10.793	.0001

Machine learning using the CART model

The results of the risk stratification using the CART model are shown in Figure 3. In terms of freedom from AF recurrence after the last ablation session, the difference was the most significant between the patients with a CAAP‐AF score of ≤5 (the low‐risk group, n = 373) and those with a score >5 (P < .001). We further divided the patients with a CAAP‐AF score >5 into two groups, and we observed that the patients with a score >8 (the high‐risk group, n = 14) showed significantly worse AF‐free survival compared to the patients with a score of 6‐8 (the intermediate‐risk group, n = 160) (P = .023).

Figure 3

Atrial tachyarrhythmia‐free survival after final AF catheter ablation based on the CAAP‐AF score: New classification developed by machine learning. The patients were divided into subgroups by using the survival classification and regression tree (CART) model to best risk stratify them for atrial tachyarrhythmia recurrence

The baseline characteristics of the patients assigned by the risk of atrial tachyarrhythmia recurrence are shown in Table 1. All six variables included to the CAAP‐AF score differed significantly among the three groups. The mean number of ablation sessions was not significantly different among the three groups. Compared to the low‐risk group, the intermediate‐risk group was 2.5 times more likely to have AF recurrence (HR 2.456, 95% CI: 1.480‐4.076, P = .0005) and the high‐risk group was 5.6 times more likely to have AF recurrence (HR 5.598, 95% CI: 2.342‐13.378, P = .0001) (Table 4).

Table 4

Risk of atrial tachyarrhythmia recurrence after last catheter ablation by CAAP‐AF score: New classification developed by machine learning

CAAP‐AF score	Patients, n	Patients with recurrence, n	HR	95% CI	P
0‐5	373	33	1	–	–
6‐8	160	28	2.456	1.480‐4.076	.0005
9‐12	14	6	5.598	2.342‐13.378	.0001

DISCUSSION

The main findings of this study were (a) the CAAP‐AF score demonstrated acceptable performance to stratify the risk of AF recurrence in a cohort of all types of AF patients treated with different strategies including CB ablation, as shown in the original paper,11 and (b) we propose subgroups which were created by the machine learning method to more clearly risk stratify the patients: low‐risk (score 0‐5), intermediate‐risk (score 6‐8), and high‐risk (score 9‐13).

Several scoring systems have been proposed to stratify the risk of AF recurrence after catheter ablation. Those include the APPLE score13 (age >65 years, persistent AF, impaired estimated glomerular filtration rate [eGFR; <60 mL/min/1.73 m2], left atrial diameter [LAD] ≥43 mm, left ventricular ejection fraction [LVEF] <50%); the ALARMc score14 (AF type, LA size, renal insufficiency, metabolic syndrome, and cardiomyopathy), the BASE‐AF2 score15 (body mass index [BMI] >28 kg/m2, atrial dilatation >40 mm, current smoking, early recurrence, AF duration >6 years, and AF type), and the MB‐LATER score16 (male, bundle branch block, LAD ≥47 mm, type of AF, and early recurrent AF). However, it seems that all four of these scoring systems have significant limitations for their application in daily practice. The study sample sizes of the studies reporting the ALARMc, BASE‐AF2, and MB‐LATER scores were small; 213, 236, and 133 patients, respectively. The APPLE score was created from 1,406 samples, but the median follow‐up period was <12 months.

In contrast, the CAAP‐AF score was created from over 2000 patients with a median follow‐up period of 1.8‐2.5 years. In addition, the CAAP‐AF score was derived from a retrospective analysis of the development cohort of 1125 patients, and the score was further validated prospectively in a test cohort including 937 patients.11 The components of the CAAP‐AF score are easy to define. Accordingly, we evaluated the CAAP‐AF score as a reliable scoring system to predict long‐term freedom from AF after catheter ablation.

However, there are several points that should be addressed when the CAAP‐AF score is going to be used in clinical practice. The score is based on data from a single center. Ablation strategies vary depending on the institute, although PV isolation should be the primary strategy everywhere.1 The development cohort consisted of patients treated between 2003 and 2010, when CB ablation was unavailable. Thus, in our present study, we tried to validate the CAAP‐AF score by using a cohort that included all eligible patients and both RF and CB ablations to determine whether the CAAP‐AF score is universally effective. To the best of our knowledge, this is the first study to confirm the reproducibility of the CAAP‐AF score.

There is at present only one study available to test the CAAP‐AF score.17 Sanhoury et al studied a group of 283 patients who were treated by CB ablation. The study successfully demonstrated the usefulness of the CAAP‐AF score for the prediction of AF recurrence after a single session. Their results are generally in line with our findings. However, the following limitation might exist; the Sanhoury et al study included predominantly patients with PAF. Since non‐PAF is one of the key components of the CAAP‐AF score, the lack of nonparoxysmal AF patients may have caused bias. In this regard, our cohort of 583 patients has much in common with the original cohort, except for the eras during which the patients were recruited and the centers where the catheter ablation procedures were done.

Another limitation of the original CAAP‐AF score may be that there were several cross‐overs among the survival curves of patients divided by the score in the test cohort.11 We observed a similar phenomenon in our cohort (Figure 1). The cross‐overs were observed in the intermediate score groups likely because the clinical characteristics of the patients in those groups were so varied that it was difficult to risk stratify those patients by CAAP‐AF scoring. Because of this limitation, in the intermediate score groups, it would have made little sense to discuss the differences in the clinical outcomes by CAAP‐AF score. The establishment of simple models in which the patients are classified into several subgroups based on the original CAAP‐AF score is expected, making it easier for clinicians in daily clinical practice to predict the chance of arrhythmia recurrence. The machine learning methods, with the ability to leverage all available data and their complex relationships, can improve both discrimination and the range of prediction over traditional statistical techniques. Among them, the CART model is relatively easy to interpret as it can graphically present the results; it is thus an emerging method for risk stratification.18

By using the CART model, we divided the patients into three categories: low‐risk (the CAAP‐AF score 0‐5), intermediate‐risk (score 6‐8), and high‐risk (score 9‐13) groups (Figure 3). Nearly 90% of the low‐risk patients were AF‐free at 2 years after the last ablation session, whereas <50% of the high‐risk patients remained AF‐free at 2 years. The prognosis of the intermediate‐risk patients was in between those of the low‐ and high‐risk patients. The information gained in the present study is surely clinically relevant. Since AF catheter ablation is costly and has a risk of serious procedure‐related complications, each patient's risk status based on the CAAP‐AF score should be taken into consideration when the decision is made whether to perform catheter ablation for AF.

LIMITATIONS

There are some limitations to this study. Compared to the original cohort, our sample size was relatively small. In addition, this study was a retrospective assessment, although the registration and systematic follow‐up for catheter ablation at our hospital were the essential basis for the study analysis. Forty‐two patients who had atrial tachyarrhythmia recurrence after the initial ablation did not undergo repeated sessions, which might have affected the results of the study since the primary endpoint of the study was the recurrence after the last session. The ablation strategy, a factor that may influence the outcome of catheter ablation especially in patients with LSPEF, is not accounted for in the CAAP‐AF score. This may be a limitation of the original CAAP‐AF score. Although we have developed a new risk stratification model using a CART analysis, the model has not been validated in another cohort.

CONCLUSIONS

The CAAP‐AF score can be used to stratify the risk of AF recurrence in patients undergoing the catheter ablation of AF. The patients can be divided into three groups: low risk (CAAP‐AF score 0‐5), intermediate risk (score 6‐8), and high risk (score 9‐13). The patient's risk status should be kept in mind when the decision is made regarding whether to perform catheter ablation for AF.

CONFLICT OF INTEREST

The authors declare no conflicts of interest for this article.