Choice of CTO scores to predict procedural success in clinical practice. A comparison of 4 different CTO PCI scores in a comprehensive national registry including expert and learning CTO operators.

BACKGROUND: We aimed to compare the performance of the recent CASTLE score to J-CTO, CL and PROGRESS CTO scores in a comprehensive database of percutaneous coronary intervention of chronic total occlusion procedures.
METHODS: Scores were calculated using raw data from 1,342 chronic total occlusion procedures included in REBECO Registry that includes learning and expert operators. Calibration, discrimination and reclassification were evaluated and compared.
RESULTS: Mean score values were: CASTLE 1.60±1.10, J-CTO 2.15±1.24, PROGRESS 1.68±0.94 and CL 2.52±1.52 points. The overall percutaneous coronary intervention success rate was 77.8%. Calibration was good for CASTLE and CL, but not for J-CTO or PROGRESS scores. Discrimination: the area under the curve (AUC) of CASTLE (0.633) was significantly higher than PROGRESS (0.557) and similar to J-CTO (0.628) and CL (0.652). Reclassification: CASTLE, as assessed by integrated discrimination improvement, was superior to PROGRESS (integrated discrimination improvement +0.036, p<0.001), similar to J-CTO and slightly inferior to CL score (- 0.011, p = 0.004). Regarding net reclassification improvement, CASTLE reclassified better than PROGRESS (overall continuous net reclassification improvement 0.379, p<0.001) in roughly 20% of cases.
CONCLUSION: Procedural percutaneous coronary intervention difficulty is not consistently depicted by available chronic total occlusion scores and is influenced by the characteristics of each chronic total occlusion cohort. In our study population, including expert and learning operators, the CASTLE score had slightly better overall performance along with CL score. However, we found only intermediate performance in the c-statistic predicting chronic total occlusion success among all scores.

Background

The percutaneous coronary intervention (PCI) of a Chronic Total Occlusion (CTO) is currently one of the most complex procedures in interventional cardiology. Compared with non-CTO PCI, interventions in chronically occluded vessels take more time, toolbox resources, radiation exposure and risk of complications [1-3]. Therefore, the patients should have a comprehensive pre-procedural assessment, including symptoms, ischemia and viability testing, in order to make a straightforward clinical indication. Currently, patients are derived for CTO PCI to improve patient’s symptoms, to reduce significant ischemia burden or to seek complete revascularization to improve left ventricular ejection fraction (LVEF) [2, 4].

Once the clinical indication is established, the operator must have a realistic estimation of the probability of procedural success that will eventually be discussed with the patient and used for clinical decision-making. The procedural difficulty is largely dictated by structural characteristics of the CTO, the coronary anatomy and clinical factors. Several scores have been developed over the last ten years to integrate these variables and perform an objective assessment of procedural CTO difficulty. Following the Multicenter CTO registry in Japan (J-CTO) score [5], the Clinical and Lesion-related (CL) [6] and Prospective Global Registry for the Study of Chronic Total Occlusion Intervention (PROGRESS CTO) scores [7] were developed and tested in study populations. More recently, the CASTLE score has been proposed on the grounds of a number of reasons [8]. First, its derivation dataset is by far the largest (14,882 patients from the EuroCTO registry compared to 329, 1143 and 521 patients in J-CTO, CL and PROGRESS scores, respectively). Second, it is representative of a large number of different European centres and operators encompassing the wholes spectrum of CTO-PCI approaches. And third, it’s the most recent and thus might reflect the impact of novel contemporary devices in CTO recanalization success.

In this study we aim to compare the performance of the new CASTLE score to the previous and representative J-CTO, CL and PROGRESS CTO scores using an extensive database of CTO-PCI procedures (the Iberian Registry of CTO PCI, or REBECO). In brief, the Association of Interventional Cardiology of the Spanish Society of Cardiology prompted an open initiative to gather prospective CTO PCI data across Spain. This ongoing Registry involves centres with a variety of expertise in CTO PCI with a pragmatic character intending to record every CTO attempt in daily practice into a real-world data-set.

Methods

The methodology, variable definitions and first results of the Iberian Registry of CTO PCI are discussed elsewhere [9]. For the present analysis (n = 1626 CTO cases), we used a data extraction from 24 centers taken August 31, 2019 (cases belong to the timeframe 2015–2019). From this database, we selected those cases with valid values on all critical variables used to estimate the four scores plus the variable CTO technical success (n = 1342 CTO cases). Technical success was defined as CTO recanalization with final TIMI 3 flow. Subsequently, the J-CTO, CL, PROGRESS and CASTLE scores were independently calculated from the raw registry data (Table 1 summarizes the score definitions of each score taken from the original publications [5-8]). Each score was dichotomized in simple or complex cases for secondary analysis using cutoffs chosen from the clinical practice [10] (CASTLE <4 vs ≥4, J-CTO <3 vs ≥3, PROGRESS CTO <3 vs ≥3, CL-SCORE <5.5 vs ≥5.5).

Table 1

Scoring system for each of the scores used in this study.

SCORE CATEGORIES	CASTLE 7 (0 to 6)	J-CTO 6 (0 to 5)	PROGRESS CTO 5 (0 to 4)	CL 15 (0 to 8 by 0.5)
CABG history	CABG history (yes)			CABG history‡ (yes)
MI history				MI history (yes)
Age	≥70
Stump	Blunt or invisible	Blunt	Poor cap visualization or non-tapered stump	Blunt
Tortuosity	Severe (≥2 bends >90° or 1 bend >120°) or unseen	1 Bending >45°	Moderate or Severe (2 bends>70° or 1 bend>90°)
Long lesion	≥20 mm (visual estimation)	≥20 mm (visual estimation)		≥20 mm† (visual estimation)
Calcification	Severe (≥50% CTO segment)	Presence of any calcification		Severe‡ (out of 3 categories)
Redo		Yes
Interventional collaterals			Absence
CTO Location			Circumflex	Non-LAD

Each item scores 1 point except (†) 1.5 points and (‡) 2 points. Definitions as per original references [5–8]. CABG, Coronary Artery Bypass Graft. CTO, Chronic Total Occlusion. MI, Myocardial Infarction. LAD, Left Anterior Descending

Qualitative variables were summarized by frequency distribution, and quantitative variables as mean values and SDs. Continuous, non-normally distributed variables were expressed as medians and interquartile ranges (IQR). Chi-Square linear p for trend was estimated for observed success rates across strata of the four scores. Hosmer-Lemeshow goodness-of-fit test, obtained by univariate logistic regression with the success rate as the dependent variable, was used to assess the calibration of the scores. Discrimination was analyzed with the area under the curve (AUC) of receiver-operator characteristics (ROC) curve. Comparison of the AUC, taking CASTLE AUC as a reference, from receiver-operating characteristic curve analysis was performed with the DeLong method [11] and p-values were corrected by Bonferroni method. To assess discrimination and reclassification ability, each score was compared with the CASTLE score as a reference by absolute integrated discrimination improvement (IDI) index, as well as continuous net reclassification index (NRI) [12]. Sensitivity, specificity, positive and negative predictive values for technical success were calculated for the complex case cutoffs previously defined by expert consensus and for the highest Youden index values. Statistical analysis was performed with STATA version 15.0 and IBM SPSS Statistics Version 21.0 (IBM Corporation, Chicago, Illinois). A 2-tailed p-value of <0.05 was considered statistically significant.

Results

Clinical and interventional characteristics of the 1,342 patients included in the study are shown in Tables 2 & 3. Mean score for CASTLE was 1.60±1.10; J-CTO 2.15±1.24; PROGRESS 1.68±0.94 and CL 2.52±1.52. The overall success rate in the patients included in the study was 77.8%. Fig 1 shows the scoring distribution among the study sample compared to each score’s original derivation cohort [5-8], except for CL score that provided only aggregated data. CL score original distribution was 33.07% score 0–1, 37.27% score 1.5–2.5, 25.55% score 3.5–4.5 and 4.11% score ≥5 compared with the Iberian Registry 25.6%, 36.2%, 30.5% and 7.7% in the same categories. The distributions show that the Iberian Registry cohort has lower complexity than CASTLE derivation cohort but higher than J CTO, PROGRESS or CL derivation cohorts. Fig 2 shows the predicted and observed success rates per each possible score’s category (obtained by logistic regression analysis). All scores showed a trend towards an inverse linear relationship between procedural success rate with score values (p<0.001 in all scores), but in the upper range of predicted scores the actual success rate was higher than expected, irrespective of the employed score. On the contrary, lower score values overestimated the actual success rate. The CASTLE and CL scores were well-calibrated using the Hosmer-Lemeshow goodness-of-fit test (p>0.05), but not the PROGRESS and J-CTO scores (p<0.05). Note that although in the highest complexity strata of CASTLE and CL scores (5–6 and 7–8 points respectively) the success rates were higher than expected, they represent <1% of the study population (see Fig 1).

Table 2

Clinical characteristics.

Variable
Age	65.17 ± 11.11
Sex (male)	84.7%
Hypertension	68.26%
Dyslipidemia	67.56%
Diabetes	35.39%
Smoker	42.70%
Creatinin (mg/dl)	1.16 ± 3.77
Chronic kidney disease	11.67%
Previous CABG	6.79%
Previous PCI	49.18%
Previous MI	29.62%
Previous stroke	5.70%
Peripheral vascular disease	10.26%
Multivessel coronary disease	58.05%
Syntax Score	17.3 ± 12.58
LVEF (%)	49.42 ± 17.54
CTO location
Left main	0.22%
Left anterior descending	32.56%
Left circumflex	16.84%
Right coronary artery	50.37%

Table 3

PCI procedure characteristics.

Variable
Redo attempt	13.93%
Primary CTO approach
Antegrade	80.5%
Retrograde	6.6%
Hybrid	11%
Unknown	1.9%
Successful technique for wire crossing#
Wire escalation	54.7%
Parallel wire / see saw	5.5%
CART or reverse CART	4.8%
Balloon-assisted reentry	2%
Not disclosed#	33%
Intravascular Ultrasound use	14%
Drug eluting stent (vs. bare metal)	99.1%
Total stent length	41.8 ± 33.1
Stent diameter	2.8 ± 0.44
Periprocedural complications	5.20%
Cardiac tamponade	0.89%
Myocardial infarction	0.75%
Perforation without tamponade	0.67%
Vascular access	0.52%
Heart Failure	0.37%
Coronary dissection (remote to CTO segment)	0.37%
Death	0.22%
Septal hematoma	0.15%
Life-threatening arrhythmia	0.15%
Others	1.12%
Contrast (ml)	254 ± 175.74
Fluoroscopy time (min)	38.19 ± 37.31
Radiation (mGy)	2419 ± 2107.91

Data are percentages or mean±standard deviation. CABG, Coronary Artery Bypass Graft. CTO, Chronic Total Occlusion. LVEF, Left Ventricular Ejection Fraction. MI, Myocardial Infarction. PCI, Percutaneous Coronary Intervention. mGy, MiliGrays. # This was an elective variable in the database

Fig 1

Barchart of scoring distribution.

The study population scoring distribution (n = 1342 CTOs) is shown in blue. Original derivation cohort scoring distribution is shown in gray, but for CL score (not available in original publication). Note that 0.5 and 7.5 are not possible to be obtained in CL-score.

Fig 2

Expected success rates versus observed success rates across different strata of each score.

P values for linear trend and Hosmer-Lemeshow (HL) tests are provided.

The discrimination of the scores for procedural success was tested using the AUC of the ROC curve (Fig 3). AUC from PROGRESS was significantly lower than CASTLE AUC, but J-CTO and CL score AUC were not different from CASTLE AUC. However, the overall discriminatory capacity of all scores was limited (as a consensus AUC <0.7 is considered poor to moderate discrimination [13]). Comparing CASTLE to J-CTO, we found no differences in reclassification abilities as evaluated with IDI /NRI (Table 4); however, CASTLE was superior to PROGRESS (IDI +0.036, p<0.001 and NRI +0,379, p<0.001). Finally, CASTLE had slightly inferior IDI than CL score (– 0.011, p = 0.004), but similar NRI (p = 0.31).

Fig 3

ROC curve and AUC of each score discriminating procedural success.

Comparison between AUCs were done taking CASTLE score as a reference.

Table 4

Integrated Discrimination Improvement Index (IDI) and Net Reclassification Improvement (NRI) of JCTO, PROGRESS and CL scores compared to CASTLE score as a reference.

	IDI	Continuous NRI (overall)	Continuous NRI event (%)	Continuous NRI nonevent (%)
JCTO	0.00496 (p = 0.335)	-0.00296 (p = 0.964)	8.43	-8.72
PROGRESS	0.03636 (p<0.001)	0.379 (p<0.001)	17.82	20.13
CL	-0.01121 (p = 0.004)	-0.0662 (p = 0.313)	12.84	-19.46

Table 5. shows sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Youden index values at “complex CTO” cutoffs (Table 5A). We also estimated these values at best Youden index value cutoffs (Table 5B) which were CASTLE <2, J-CTO <3, PROGRESS <2, CL ≥2.5.

Table 5

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and Youden index values at different cutoffs.

Score	Cutoff value	Sensitivity	Specificity	PPV	NPV	Youden index
A. For binary (simple vs complex) CTO cutoffs
CASTLE	<4	96.26%	11.41%	79.20%	46.58%	0.08
J-CTO	<3	64.85%	57.72%	84.31%	31.91%	0.23
PROGRESS	<3	81.32%	23.83%	78.90%	26.69%	0.05
CL	<5.5	95.79%	13.42%	79.49%	47.62%	0.09
B. For maximal Youden index values cutoffs
CASTLE	<2	55.56%	66.44%	85.29%	29.91%	0.22
J-CTO	<3	64.85%	57.72%	84.31%	31.91%	0.23
PROGRESS	<2	43.87%	68.46%	82.97%	25.82%	0.12
CL	<2.5	50.67%	72.48%	86.58%	29.55%	0.23

Discussion

This study provides a comprehensive comparison of the new CASTLE score against the most commonly used CTO scores in an extensive, national database of CTO procedures, including expert and learning CTO operators. However, we found poor-to-intermediate performance in the c-statistic predicting CTO success among all scores. With small differences, CASTLE score performed best along with CL score, followed by J-CTO and PROGRESS with slightly worse efficiency. An interesting lesson of this study is that applying different scores to the same cohort, the spectrum of difficulty is variable and not the same as in the original CTO scores cohorts (Fig 1). In other words, there seems to be a lack of consistency among them. How, then, a CTO score should be chosen?

The probability of success in CTO PCI is dependent on multiple factors. On the one hand, on the operator’s experience and the availability and use of an extensive toolbox. On the other, it relies on the use of a structured approach, in which the use of CTO PCI scores plays an essential role [2, 14]. First, they allow the operator to gauge the feasibility of procedural success according to his/her level of expertise, particularly over the learning curve of CTO PCI. Second, they facilitate Heart Team discussions in cases in which CTO lesions are critical targets in an achieving equivalent degree of myocardial revascularization. And third, they provide insights on the procedural time, amount of contrast, radiation dose and risk of complications associated with the intervention that can be integrated into the decision-making process and in PCI planning [14].

CASTLE is the CTO score derived from the largest dataset (14,882 patients taken from 2008 to 2014), encompassing a broader number of operators, techniques and practices across Europe. Some relevant differences compared to previous scores must be pointed out. The J-CTO was derived from a multi-centre Japanese database comprising 400 procedures (2006–2007), and designed to estimate the likelihood of passing an antegrade guidewire in less than 30 minutes [5]. Posteriorly, it was extensively validated as success and even outcomes predictor [15, 16]. The CL score was designed to assess procedural failure in a first CTO-PCI procedure, including for the first-time clinical variables. However, it was derived from a single-centre European cohort of mainly (90.7%) antegrade CTO cases (n = 1,671, from 2004 to 2013) [6]. PROGRESS CTO was derived from a multi-centre US database, is more contemporary to CASTLE (2012–2015), and designed to assess technical success using the hybrid approach [7]. These three were chosen as comparators because they are the most commonly used scores and represent different regional approaches.

The heterogeneous derivation cohorts probably explain that the different scores include a heterogeneous set of variables, except for the blunt proximal cap (Table 1). Two scores include clinical variables. They concur in CABG, which is acknowledged as an adverse feature in CTO patients [17, 18]. Moreover, a recent study from the PROGRESS database found 5% less recanalization success in CABG patients compared to non-CABG patients [19]. CTO characteristics such as tortuosity and calcification are more challenging to be consistently evaluated, as they are defined discordantly among scores and might be somewhat subjective to the operator. However, we agree that differently to the softer J-CTO had definitions [5], severe tortuosity and severe calcification are common ground, especially in combination, for challenging CTO scenarios. Finally, it is remarkable that only one score (PROGRESS) includes assessing collaterals in the scoring, which is very relevant for CTO planning. Despite these differences, we genuinely believe that the careful CTO evaluation needed to calculate one or more scores is valuable for procedural planning, especially for the less experienced operator.

Comparing our cohort with the derivation cohorts of previous scores we see a shift towards higher complexity in the REBECO Registry compared to older J CTO, CL and PROGRESS scores (Our cohort starts at year 2015 while J CTO, CL and PROGRESS have older derivation cohorts [5-7]). This is probably as a result of widespread standardization of techniques and equipment allowing CTO operators to tackle more complex cases [2]. In contrast, the pragmatic nature of the Iberian Registry includes centres and operators at different points of the CTO learning curve, quite differently from, for example, the CASTLE data derived from the highly expert EuroCTO Club [8]. Consequently (and saving the differences in endpoints), we had less recanalization success (77.8%) compared to 84.2% in CASTLE, 92.5% in PROGRESS and 88.6% in J CTO; but better than the CL cohort (72.5%) [5-8].

Many of the newly developed scores compared themselves in the original reports to J-CTO, showing better parameters of calibration and discrimination [5-8]. However, this calculation might be biased because the validation and derivation cohort come together from a single “mother” cohort that is likely to perform better with its derived score than with an external score (J-CTO). Some score comparisons have been previously published showing that the performance of the scores might be similar. Karastakis et al. compared CL, J-CTO and PROGRESS scores in a cohort from the PROGRESS CTO registry (n = 664), showing similarly poor to moderate (<0.7) discrimination as evaluated per AUC (with no inter-score differences) [20]. This analysis might be biased because the study sample also comes from the PROGRESS CTO Registry. Recently, Kalogeropoulos et al. used an international database (n = 660) to compare CASTLE to J-CTO, finding equal overall discriminatory capacity of both scores (AUC 0.676 and 0.698 respectively). CASTLE outperformed J-CTO in the most complex cases (J-CTO ≥3 or CASTLE ≥4 representing only roughly 9% of the sample) but with quite low overall AUC (0.588). Our comparison study has the strengths of a more extensive, independent testing database (n = 1,342) and a more comprehensive analysis using four scores and taking CASTLE score as a reference.

In our study, the calibration (meaning how close the observed and expected results were) was better for CASTLE and CL scores, although this test does not allow for inter-score comparisons. However, the differences are rather small (Fig 2). The discrimination measured with AUC and with the IDI index was better for CASTLE, J-CTO and CL and slightly worse for PROGRESS score; however poor to moderate (<0.7) in absolute terms, in agreement with previous publications [10, 20]. Complementary to the AUC that has some limitations [21], we assessed the incremental value of the newer CASTLE score using two reclassification indexes, the Integrated Discrimination Improvement (IDI) and the Net Reclassification Improvement (NRI) indices [12, 22]. The IDI, which is possibly more sensitive than the AUC comparison showed that CASTLE had better discrimination than PROGRESS, similar to J-CTO and slightly inferior than CL. The NRI analysis showed that CASTLE reclassified cases better than PROGRESS in roughly 20% of cases (CASTLE correctly reclassified 17.82% of event cases and 20.13% of nonevent cases into a higher or lower predicted risk of success, respectively). However, CASTLE did not significantly improve reclassification compared to J-CTO and CL scores. The intermediate or poor performance of current available CTO scores in predicting CTO success suggests the need for more precise mechanisms to predict the outcomes and precisely inform our patients.

We provided in Table 4 data on sensitivity, specificity, PPV, NPV and Youden indexes showing only modest values with commonly used complex CTO cutoffs and cutoffs that maximizes the Youden index. However, we believe that binary categorization is unnecessary in CTO scores because it masks the spectrum of complexity provided by scores. Furthermore, PPV and NPV provide insufficient precision to inform for or against attempting a specific CTO case (highly expert operators demonstrated high J-CTO scores being non-associated with observed success rates [23]).

The information provided by our study may also help in selecting a specific CTO PCI score for particular purposes. Very experienced operators with success rates over 90% will take little interest on the success discrimination capabilities but might use CASTLE or CL in order to discuss efficiency and complication risks with patients (both have the advantage of combining angiographic and clinical variables; CASTLE is probably more intuitive to calculate and has fewer categories than CL score). Less experienced operators might choose CASTLE or CL scores on the grounds of better calibration and discrimination to predict which highly complex cases might benefit for proctoring or referral (although we must bear in mind that in this or any other study the overall discrimination is poor to moderate). Also, the predicted success rates might be easily obtained with univariate logistic regression analysis in a local database, using one or more scores to choose the one with the best “personalized” calibration. For research and benchmarking, J-CTO is the oldest and most widespread score and thus is critical to allow comparison with earlier studies. CASTLE is without a doubt the score with more solid foundations in terms of contemporaneity and derivation dataset size, so it will be probably a reference for future publications.

A few limitations should be reported regarding this study. First, the original angiographies were not assessed by a central core lab; we trusted in the individual investigator’s evaluation of each item contributing to the different scores. Second, the collected data is self-reported and not systematically audited, so in spite of quality control some degree of selection and reporting bias are possible. Third, although this is a multi-centre, contemporary database, it might not be representative of specific practices, strategies or skillsets. Finally, more scores might be considered for comparison, although as discussed before, we found these as the most commonly used scores.

Several available CTO scores were not assessed in our study on the grounds of the preferential use of a specific device or strategy (CrossBoss and hybrid techniques in Europe, RECHARGE registry [24]); derivation from a single operator’s experience (ORA score [25]); or the need for interesting but non-mandatory methods in CTO assessment (CT-RECTOR [26] or KCCT [27] scores). However, many predictors of procedural failure are common: stump, calcification, tortuosity, length, previous CABG.

Conclusion

We compared 4 CTO recanalization success scores in a large, independent, multicenter database. Overall discrimination was poor to moderate with c-statistics predicting CTO success below 0.7 among all scores. CASTLE score performed best along with CL score, followed by J-CTO and PROGRESS with slightly worse efficiency.

Procedural PCI difficulty is not consistently depicted by available CTO scores and is probably influenced by the characteristics of each CTO cohort. Operators in different points of their learning curve should be aware and consider the choice of an adequate score for a specific purpose. In the case of our study population, including expert and learning operators, the CASTLE score had slightly better overall performance along with CL score.

15 Dec 2020

PONE-D-20-30053

Choice of CTO scores to predict procedural success in clinical practice. A comparison of 4 different CTO PCI scores in a comprehensive national registry including expert and learning CTO operators.

PLOS ONE

Dear Dr. Salinas,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Reviewers have raised concerns -among other issues- about the lack of data reporting the relative use of different techniques to achieve vessel recanalization in order to appreciate the actual complexity of the cases included and how to account for different definitions across the diverse scores. Reviewers have highlighted that the value of scores should be put in perspective according to the skills and experience of the operators.

Please submit your revised manuscript by Jan 28 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Giuseppe Andò, M.D., Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. One of the noted authors is a group or consortium [REBECO collaborators]. In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Everybody knows the importance to calculate scores in order to quickly understand the complexity of the intervention, but we also know that complexity is proportional to the operators expertise and clinical comorbidities. For this reason it would be desirable to use the best score just for learning operators. This is a real world comparison between different scores in unselected patients underwent CTO recanalization by expert and learning CTO operators. It was interesting to read this manuscript. I have nothing to add. Well done.

Reviewer #2: In this paper the authors wanted to compare 4 CTO scores to predict the success in CTO PCI in the Spanish National Registry (REBECO). This regitry included expert and learning CTO operators, including more than 1300 patients treated in the period 2015-2019.

The 4 scores have different anatomical variables that have been integrated in order to perform an objective assesment of procedural CTO difficulty. The most recent score is the CASTLE score, developed from EuroCTO database, with more than 14.000 patients included. Other three scores were: J CTO score (Japan), Progress CTO score (USA), CL score (France)

The statistical evaluation in the paper considered the comparison between Spanish registry and the 4 scores in therms of calibration, discrimination and reclassification.

The statistical analysis was very well developed and conducted, resulting in a slightly better overal performance of CASTLE score , but with only a poor to intermediate performance in the prediction of CTO PCI success among all scores.

The authors concluded that the probability of success in these procedures depends on multiple factors, and the most important is operator's expertise.

I think that CTO scores are really important for many reasons especially for operators at the beginning of their learning curve, because in that group of operator the correlation of CTO score complexity and success rate can be important. One young operator with success rate near 70%, can have a great help in selectong cases that are not so complex in order to reduce the risk of complication and increase their success rate.

On the contrary in expert operators, with a success rate more than 90% overall, the use of a CTO score is less importan, because with the modern approach to CTO PCI, hybrid approach, one operator needs to be flexible during the procedure and he is able to change many techniques during the same procedure, also in complex cases with high CTO scores.

Moreover the use of CTO scores is helping the operator in the most important phase of the procedure, that means the pre.procedural phase of planning and careful evaluation of the anatomical features of the CTO lesion (lenght, calcium, tortuosity, prox cap ambiguity, presence of interventional collaterals).

Then less experienced operator can evaluate with CTO scores which complex patients might benefit for proctoring or referred to a more experienced operator.

The authors conclusion is appropriate with the data presented in the paper .

Reviewer #3: The authors used a national CTO database to compare the performance of 4 different CTO success prediction scores (CASTLE, J-CTO, PROGRESS and CL). They concluded that the CASTLE and CL scores had a slightly better performance overall, but the predicting ability of all scores was at most moderate.

Up-take of CTO PCI increases and therefore the authors’ attempt to identify the most accurate score, which would facilitate the multiple aspects of CTO PCI decision-making and procedural planning, is interesting and clinically relevant. The study is well conducted and the manuscript well written.

However, I have the following comments/queries:

- The authors do not report the successful mode of vessel recanalization (Antegrade Wire Escalation vs. Antegrade Dissection Re-entry vs. Retrograde approach). Therefore, the adoption of advanced contemporary CTO techniques in the described cohort is unknown. This information is important in order to appreciate the complexity of the cases, the skills of the operators involved and to put the described cohort in a contemporary and comparable setting. Furthermore, other essential procedural information (e.g. stent length and size) are missing.

- Although, the authors briefly discuss the derivation cohorts for the 4 scores, they do not discuss at all their different clinically relevant variables. For example, it is common knowledge among experienced CTO operators that CTO PCI in previous CABG cases (a scoring point for CASTLE and CL) is more challenging. On the other hand, how accurate can a score be when it does not include an assessment of collateral channels and the retrograde approach is the successful strategy in up to 1/3 of the cases in contemporary expert practice? The above discussion is an important part of any manuscript examining CTO PCI predictions scores.

- Tortuosity and calcification have different definitions in different scores. How was this problem addressed during score calculation since the angiograms were not re-assessed?

- The REBECO registry has a voluntary character and the data collected are self-reported. Is there a quality/validity assessment process for the collected data? Selection and reporting bias should be recognised as a limitation of the study.

- According to the authors the registry includes expert and learning operators. An analysis based on the level of experience would be interesting.

- The authors report periprocedural complications with a rate of 5.2%. Is there a breakdown for these complications?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Roberto Garbo, MD

Maria Pia Hospital

GVM Care & Research

Turin, Italy

Reviewer #3: Yes: Dr Grigoris Karamasis

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

4 Jan 2021

Editorial comments

Comments to the Author

1) Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.

Reply: The R1 manuscript was thoroughly revised and adapted to PLOS ONE's style requirements, including those for file naming.

2) One of the noted authors is a group or consortium [REBECO collaborators]. In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

Reply: Regarding the research consortium 'REBECO Group' to which we belong, we include the full list of investigators in the acknowledgements, instead of a supplementary file (therefore there are no supplementary files now in the article). We respectfully ask the editorial board to include the collaborator/investigator names that do not qualify as article's authors in the article by-name to be indexed in Medline as such (https://www.nlm.nih.gov/bsd/policy/authorship.html). Therefore, we included the investigator names on the first page (as well as in acknowledgements).

The leading investigator of REBECO group is Jose Antonio Fernández (joseantoniofer@gmail.com). This information is given in the acknowledgements.

3) Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Reply: As commented before there are no longer supporting Information files

Reviewer #1

1) Everybody knows the importance to calculate scores in order to quickly understand the complexity of the intervention, but we also know that complexity is proportional to the operators expertise and clinical comorbidities. For this reason it would be desirable to use the best score just for learning operators. This is a real world comparison between different scores in unselected patients underwent CTO recanalization by expert and learning CTO operators. It was interesting to read this manuscript. I have nothing to add. Well done.

Reply: Thank you for your kind words on our manuscript

Reviewer #2:

1) In this paper the authors wanted to compare 4 CTO scores to predict the success in CTO PCI in the Spanish National Registry (REBECO). This regitry included expert and learning CTO operators, including more than 1300 patients treated in the period 2015-2019.

The 4 scores have different anatomical variables that have been integrated in order to perform an objective assesment of procedural CTO difficulty. The most recent score is the CASTLE score, developed from EuroCTO database, with more than 14.000 patients included. Other three scores were: J CTO score (Japan), Progress CTO score (USA), CL score (France)

The statistical evaluation in the paper considered the comparison between Spanish registry and the 4 scores in therms of calibration, discrimination and reclassification.

The statistical analysis was very well developed and conducted, resulting in a slightly better overal performance of CASTLE score , but with only a poor to intermediate performance in the prediction of CTO PCI success among all scores. The authors concluded that the probability of success in these procedures depends on multiple factors, and the most important is operator's expertise.

I think that CTO scores are really important for many reasons especially for operators at the beginning of their learning curve, because in that group of operator the correlation of CTO score complexity and success rate can be important. One young operator with success rate near 70%, can have a great help in selectong cases that are not so complex in order to reduce the risk of complication and increase their success rate.

On the contrary in expert operators, with a success rate more than 90% overall, the use of a CTO score is less importan, because with the modern approach to CTO PCI, hybrid approach, one operator needs to be flexible during the procedure and he is able to change many techniques during the same procedure, also in complex cases with high CTO scores. Moreover the use of CTO scores is helping the operator in the most important phase of the procedure, that means the pre.procedural phase of planning and careful evaluation of the anatomical features of the CTO lesion (lenght, calcium, tortuosity, prox cap ambiguity, presence of interventional collaterals).

Then less experienced operator can evaluate with CTO scores which complex patients might benefit for proctoring or referred to a more experienced operator.

The authors conclusion is appropriate with the data presented in the paper .

Reply: Thank you for your nice words on our study. We agree with your opinion, and we added some of your interesting thoughts in the discussion:

Despite these differences, we genuinely believe that the careful CTO evaluation needed to calculate one or more scores is valuable for procedural planning, especially for the less experienced operator.

Reviewer #3

The authors used a national CTO database to compare the performance of 4 different CTO success prediction scores (CASTLE, J-CTO, PROGRESS and CL). They concluded that the CASTLE and CL scores had a slightly better performance overall, but the predicting ability of all scores was at most moderate.

Up-take of CTO PCI increases and therefore the authors’ attempt to identify the most accurate score, which would facilitate the multiple aspects of CTO PCI decision-making and procedural planning, is interesting and clinically relevant. The study is well conducted and the manuscript well written.

However, I have the following comments/queries:

1) The authors do not report the successful mode of vessel recanalization (Antegrade Wire Escalation vs. Antegrade Dissection Re-entry vs. Retrograde approach). Therefore, the adoption of advanced contemporary CTO techniques in the described cohort is unknown. This information is important in order to appreciate the complexity of the cases, the skills of the operators involved and to put the described cohort in a contemporary and comparable setting. Furthermore, other essential procedural information (e.g. stent length and size) are missing.

Reply: Thank you for your interest. We limited the amount of information on technical details to focus on the statistical analyses comparing the scores. We substituted the variable: ‘Primarily retrograde or hybrid approach’ in Table 2 for a more detailed description ‘Primary CTO approach’. We also added to Table 2 the following data: ‘Successful technique for wire crossing’, ‘Intravascular Ultrasound use’, ‘Drug eluting stent (vs bare metal)’, ‘Stent length’, ‘Stent diameter’. As Table 1 became too large, we decided to break it in two pieces (clinical characteristics and interventional characteristics)

Primary CTO approach

Antegrade

Retrograde

Hybrid

Unknown

80.5%

6.6%

11%

1.9%

Successful technique for wire crossing#

Wire escalation

Parallel wire / see saw

CART or reverse CART

Balloon-assisted reentry

Not disclosed�

54.7%

5.5%

4.8%

2%

33%

Intravascular ultrasound use 14%

Drug eluting stent (vs bare metal) 99.1%

Stent length 41.8 �33.1

Stent diameter 2.8 � 0.44

2) Although, the authors briefly discuss the derivation cohorts for the 4 scores, they do not discuss at all their different clinically relevant variables. For example, it is common knowledge among experienced CTO operators that CTO PCI in previous CABG cases (a scoring point for CASTLE and CL) is more challenging. On the other hand, how accurate can a score be when it does not include an assessment of collateral channels and the retrograde approach is the successful strategy in up to 1/3 of the cases in contemporary expert practice? The above discussion is an important part of any manuscript examining CTO PCI predictions scores.

Reply: We agree with the reviewer on the heterogeneity of the different variables included in the scores. We added a paragraph on this issue at Discussion:

The heterogeneous derivation cohorts probably explain that the different scores include a heterogeneous set of variables, except for the blunt proximal cap (Table 1). Two scores include clinical variables. They concur in CABG, which is acknowledged as an adverse feature in CTO patients(18,19). Moreover, a recent study from the PROGRESS database found 5% less recanalization success in CABG patients compared to non-CABG patients(20). CTO characteristics such as tortuosity and calcification are more challenging to be consistently evaluated, as they are defined discordantly among scores and might be somewhat subjective to the operator. However, we agree that differently to the softer J-CTO had definitions(5), severe tortuosity and severe calcification are common ground, especially in combination, for challenging CTO scenarios. Finally, it is remarkable that only one score (PROGRESS) includes assessing collaterals in the scoring, which is very relevant for CTO planning. Despite these differences, we genuinely believe that the careful CTO evaluation needed to calculate one or more scores is valuable for procedural planning, especially for the less experienced operator.

3) Tortuosity and calcification have different definitions in different scores. How was this problem addressed during score calculation since the angiograms were not re-assessed?

Reply: This is an excellent question. As commented in the previous new paragraph, this might be a source of possible discrepancies in CTO features evaluation. Moreover, from the research standpoint, it’s very complex to host every different definition of each variable in a database. The CL score does not explicitly define the calcification (they only describe it as mild, moderate, or severe). We indeed did not re-review the angiograms (which was stated as a limitation), but we had what we believed was a good enough categorization of these variables in the REBECO database. We re-calculated the scores from the raw data using the following scheme:

Tortuosity in REBECO raw database CASTLE J CTO PROGRESS CL

No / Mild (<45º) 0 0 0 -

Moderate (≥ 45º <90º) 0 1 0 -

Severe (>90) 1 1 1 -

Calcification in REBECO raw database

No 0 0 - 0

Mild (only during cine) 0 1 - 0

Moderate (visible in fluoro <50% CTO) 0 1 - 0

Severe (visible in fluoro ≥50% CTO) 1 1 - 1

4) The REBECO registry has a voluntary character and the data collected are self-reported. Is there a quality/validity assessment process for the collected data? Selection and reporting bias should be recognised as a limitation of the study.

Reply: Yes, there is a risk of selection and reporting bias, as there is no monitoring of contributing centers and the data is self-reported. However, the Registry has the following quality controls:

1. The leading investigators of each center must be affiliated to the Spanish Interventional Cardiology Association (ACI-SEC), and therefore reports the yearly activity data to the general ACI-SEC Registry (administrative Registry with a limited set of variables such as the total number of different procedures, success rate, access, etc.), including the number of CTO procedures. The study coordinators compare the number of CTO procedures sent to the general ACI-SEC Registry and the REBECO Registry and ask the REBECO centers to reconcile the numbers in the REBECO database if there are discrepancies. If the center persists without including every CTO case, it is considered inactive.

2. The Registry is free to any center performing CTO to participate (therefore the mix of expert and learning CTO centers and the absolute number of CTO success). Still, the investigators are instructed to include consecutive cases. The Registry is open and enrolling patients, so if a center has a drop or a stop in CTO cases reporting a query is sent to this center. If the center discontinues the reporting to the database, it is deemed inactive and eventually closed for the Registry.

We added in limitations the following sentence:

Second, the collected data is self-reported and not systematically audited, so despite quality control, some degree of selection and reporting bias are possible.

5) According to the authors the registry includes expert and learning operators. An analysis based on the level of experience would be interesting.

Reply: Thank you for your interest in this matter. In the first Registry report with the first 1000 patients the success rate varied with the center’s experience (Amat-Santos IJ, Martin-Yuste V, Fernández-Díaz JA, Martin-Moreiras J, Caballero-Borrego J, Salinas P, et al. Procedural, Functional and Prognostic Outcomes Following Recanalization of Coronary Chronic Total Occlusions. Results of the Iberian Registry. Rev Esp Cardiol Engl Ed. 1 de mayo de 2019;72(5):373-82.):

In the present study, we selected those cases with valid values on all critical variables used to estimate the four scores plus the variable CTO technical success (n=1342 CTO cases). As suggested, we performed a sensitivity analysis based on the volume of cases performed by center (quartiles of CTO volume), finding similar results with non-significant differences:

We also compared AUC for CTO success comparing centers above or below the p50, and above or below the p75 of procedures, finding no relevant differences:

AUC full sample

(reported) AUC <p50 centers AUC >p50 centers AUC <p75 centers AUC >p75 centers

CASTLE 0.633 0.638 0.633 0.629 0.635

J-CTO 0.628 0.702 0.620 0.661 0.614

PROGRESS 0.557 0.601 0.552 0.567 0.554

CL 0.652 0.632 0.655 0.653 0.652

Therefore, we believe these sub-studies probably do not add value to the article, which is considerably long so far. However, we remain open at the reviewer's judgment on the possibility of adding these analyses as supplementary material.

6) The authors report periprocedural complications with a rate of 5.2%. Is there a breakdown for these complications?

Periprocedural complications

Cardiac tamponade

Myocardial infarction

Perforation without tamponade

Vascular access

Heart Failure

Coronary dissection (remote to CTO segment)

Death

Septal hematoma

Life-threatening arrhythmia

Others 5.20%

0.89%

0.75%

0.67%

0.52%

0.37%

0.37%

0.22%

0.15%

0.15%

1.12%

Reply: We added the breakdown of these complications. As the Table 1 became too large, we decided to break it in two pieces (clinical characteristics and interventional characteristics).

Submitted filename: Response to reviewers.docx

Click here for additional data file.

11 Jan 2021

Choice of CTO scores to predict procedural success in clinical practice. A comparison of 4 different CTO PCI scores in a comprehensive national registry including expert and learning CTO operators.

PONE-D-20-30053R1

Dear Dr. Salinas,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Giuseppe Andò, M.D., Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Grigoris Karamasis

24 Mar 2021

PONE-D-20-30053R1

Choice of CTO scores to predict procedural success in clinical practice. A comparison of 4 different CTO PCI scores in a comprehensive national registry including expert and learning CTO operators.

Dear Dr. Salinas:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Giuseppe Andò

Academic Editor

PLOS ONE