Diagnostic accuracy of 64-slice computed tomography coronary angiography for the detection of in-stent restenosis: a meta-analysis.

BACKGROUND: We sought to evaluate the diagnostic accuracy of 64-slice multi-detector row computed tomography (MDCT) compared with invasive coronary angiography for in-stent restenosis (ISR) detection.
METHODS: MEDLINE, Cochrane library, and BioMed Central database searches were performed until April 2009 for original articles. Inclusion criteria were (1) 64-MDCT was used as a diagnostic test for ISR, with >50% diameter stenosis selected as the cut-off criterion for significant ISR, using invasive coronary angiography and quantitative coronary angiography as the standard of reference; (2) absolute numbers of true positive, false positive, true negative, and false negative results could be derived. Standard meta-analytic methods were applied.
RESULTS: Nine studies with a total of 598 patients with 978 stents included were considered eligible. On average, 9% of stents were unassessable (range 0-42%). Accuracy tests with 95% confidence intervals (CIs) comparing 64-MDCT vs invasive coronary angiography showed that pooled sensitivity, specificity, positive and negative likelihood ratio (random effect model) values were: 86% (95% CI 80-91%), 93% (95% CI 91-95%), 12.32 (95% CI 7.26-20.92), 0.18 (95% CI 0.12-0.28) for binary ISR detection. The symmetric area under the curve value was 0.94, indicating good agreement between 64-MDCT and invasive coronary angiography.
CONCLUSIONS: 64-MDCT has a good diagnostic accuracy for ISR detection with a particularly high negative predictive value. However, still a relatively large proportion of stents remains uninterpretable. Accordingly, only in selected patients, 64-MDCT may serve as a potential alternative noninvasive method to rule out ISR.

Introduction

Coronary stent implantation is increasingly performed in the treatment of significant coronary artery disease and has significantly reduced the occurrence of restenosis as compared with balloon angioplasty.1,2 Moreover, with the recent introduction of drug-eluting-stents (DES), the occurrence of in-stent restenosis (ISR) has further decreased.3,4 Nonetheless, even after DES implantation, excessive neo-intimal hyperplasia resulting in partial or complete ISR may still occur. Accordingly, in patients presenting with recurrent chest pain following DES implantation, invasive coronary angiography remains frequently indicated to evaluate the presence of ISR. However, considering the fact that a substantial number of these invasive coronary angiograms are not followed by intervention, the need for a noninvasive alternative approach for ISR detection is evident. To this end, stress tests may be used to assess patients with suspicion of ISR. However, the diagnostic accuracy of stress tests for ISR detection is moderate,5 and direct stent visualization would be preferred. With previous 4- and 16-slice multi-detector row computed tomography (MDCT) systems, MDCT was of limited value in the assessment and follow-up of patients with coronary stents, due to the frequent occurrence of motion and blooming artefacts.6,7 With the introduction of 64-slice systems, some of these limitations have been partially overcome due to increased temporal resolution (due to increased gantry rotation speed), increased spatial resolution and enhanced craniocaudal coverage. Although initial data obtained with 64-MDCT appear promising, more robust data are needed to confirm that this technique may become a potential alternative to invasive coronary angiography for ISR detection in daily clinical practice. The most important features of a clinical test are few false negative and false positive results (affecting sensitivity and specificity, respectively). Additionally, when restenosis rate is low, as it is currently the case for simple lesions treated with DES, a high negative predictive value (NPV) allows exclude ISR in the majority of patients. Subsequently, only in a limited number of patients, invasive coronary angiography would be required to confirm and potentially treat ISR.

In order to determine the current diagnostic accuracy of 64-MDCT, we performed a meta-analysis of all available studies comparing 64-MDCT with invasive coronary angiography for the diagnosis of ISR.

Methods

Search Strategy

Database searches for English articles published until April 2009 were performed in MEDLINE, Cochrane library, and BioMed Central databases. We combined the medical subject headings for computed tomography, multi-detector computed tomography, and coronary angiography, with the exploded terms stent and restenosis and scanned references in retrieved articles and reviews. The retrieved studies were carefully examined to exclude potential duplicates or overlapping data. Meeting abstracts were excluded, as they could not provide adequately detailed data and their results might not be final. Only papers evaluating the presence of ISR by both invasive coronary angiography and 64-MDCT in the same subjects were included.

Study Eligibility

We included a study if: (1) 64-MDCT was used as a diagnostic test for ISR, with >50% diameter stenosis selected as the cut-off criterion for significant restenosis, using invasive coronary angiography and quantitative coronary angiography as the standard of reference; (2) absolute numbers of true positive (TP), false positive (FP), true negative (TN), and false negative (FN) results were provided or could be derived. Studies were excluded if they were performed: (1) only in patients after coronary artery bypass graft surgery; (2) in a subset of patients with prior heart transplantation.

Data Extraction

The following information was extracted from each study: first author, year of publication, and journal; study population characteristics, including sample size (number of subjects evaluated with both tests, number of patients excluded); number of stents evaluated (and excluded from the analysis); gender; mean age (and standard deviation); mean heart rate (and standard deviation); relative timing of the two imaging procedures and whether or not evaluation of one test was blind to the result of the other; technical characteristics of the MDCT, including type and manufacturer of MDCT equipment used, and rate of beta-blocker usage. Two investigators performed data extraction independently. Discrepancies were solved by a third investigator and global consensus. The study quality conformed to the QUADAS guidelines.8

Statistical Analysis

The main analyses were performed using the traditional meta-analytic methods for combining data for diagnostic accuracy tests.9,10 The analyses that were performed to compare accuracy of 64-MDCT vs invasive coronary angiography as reference incorporated all accuracy tests separately: sensitivity, specificity, likelihood ratios, diagnostic odds ratio, and finally summarizing data in receiver operating characteristic curves (sROC). Accordingly, the absolute numbers of TP, FP, TN, and FN findings were analyzed to provide sensitivity and specificity. Positive and negative likelihood ratios (LR+ and LR−) were calculated as LR+ = sensitivity/(1 − specificity) and LR− = (1 − sensitivity)/specificity, respectively. The LR+ is a measure of how much the odds of the disease increase when a test is positive, while the LR− is a measure of how much the odds of the disease decrease when a test is negative. Thus, the combined likelihood ratios provides the diagnostic odds ratio (=[sensitivity/(1 − specificity)]/[(1 − specificity/sensitivity] = (true positive × true negative)/(false positive × false negative)). The result is a ratio of the odds of a positive test result among diseased to the odds of a positive test result among nondiseased.9,10 Diagnostic odds ratio converts each pair of sensitivity and specificity into a single global accuracy measure.

Sensitivity and specificity were pooled as weighted means in which the weight of each study is its sample size. Likelihood ratios and diagnostic odds ratios were pooled by the DerSimonian and Laird method (random-effects model).11 The accuracy tests were repeated with inclusion of unassessable stents. The summary ROC analysis, that represents a more general method for the meta-analysis of diagnostic studies than those described above, was computed using the Moses-Shapiro-Littenberg method.12

Source of heterogeneity were explored performing a meta-regression using the Moses-Shapiro-Littenberg method with covariates12: average age of patients, percentage of male, percentage of not evaluable stents, sample size.

Meta-analysis software package Meta-disc version 1.4 (Unit of clinical biostatics, the Ramo y Cajal Hospital, Madrid, Spain) was used for analyses. Moreover, prevalence of ISR, positive, and negative predictive values were calculated as well.

Finally, to check how robust our findings were to model specification we also performed a sensitivity analysis. Rather than using a single accuracy endpoint per study we used a multivariate meta-analysis model. We used the hierarchical summary ROC model which in our setting represents a different parametrization of the bivariate random-effects meta-analysis model.13 Additional analysis was conducted using STATA/SE 10.1 (Stata Statistical Software: Release 10. College Station, TX: StataCorp LP).

Results

Search Results

The search resulted in detection of 211 hits. After the exclusion of nonrelevant articles by title and abstract, 37 articles were retrieved for full text evaluation, and of these 10 studies fulfilled all inclusion criteria. One study was excluded because it was performed using a dual-source 64-slice MDCT system.14

From the study by Rist et al15 the data on proximal, distal, and in-stent restenosis were combined to a single result. Despite using a 40-MDCT system, the study by Gaspar et al16 was included as the spatial resolution was similar to 64-MDCT. Finally, for this reason, the study by Van Mieghem et al,17 which used both 16- and 64-slice scanners, was also included in the analysis. Accordingly, nine eligible studies15-23 were included in the meta-analysis (Table 1).

Table 1

Characteristics of included studies

Study	Year	Patients (n)	Stents (n)	Stents >3 mm (%)	DES (%)	Male (%)	DM (%)	Mean age years (SD)	Mean HR bpm (SD)	β-blockers (%)	Unassessable stents (%)
Gaspar	2005	65	111	–	–	69	–	63 (12)	–	–	4.5
Van Mieghem	2006	70	162	100	95	83	19	61 (11)	57 (7)	70	0
Rixe	2006	64	102	31	30	64	–	58 (10)	60 (5)	–	42
Rist	2006	25	46	–	–	92	–	59 (12)	62 (8)	56	2.2
Ehara	2007	81	125	89	25	78	27	67 (10)	72 (13)	25	12
Oncel	2007	30	39	41	15	90	–	58 (10)	–	–	0
Cademartiri	2007	182	192	34	–	84	13	58 (11)	–	–	7.3
Carrabba	2007	41	87	56	80	90	12	68 (11)	54 (6)	70	0
Manghat	2008	40	114	62	56	90	13	64 (10)	63 (11)	–	9.6
All		598	978	62	56	81	17	61.2	62.1	52	9.1

DES Drug eluting stent, DM diabetes mellitus, SD standard deviation, HR heart rate, bpm beat per minute.

Bold values indicate summarized findings (total or weighted means).

Characteristics of the Included Studies

The mean patient age was 61 years with a range of 58-68 years. Males constituted 81% (range 63-92%), and 17% (range 12-27%) had diabetes mellitus. The mean heart rate during data acquisition (reported in six studies) was 62 beats per min with a range of 54-72 beats per min. In five studies patients received additional beta-blockers to reduce heart rate before the examination (Table 1).

Stent Assessability

The analysis included 598 patients with a total of 978 stents. The percent of DES evaluated was 56% (data available in six studies). Overall the percent of unassessable stents was 9% (range 0-42%), although in three studies all stented segments were evaluated17,19,22 (Table 1).

Results of the Accuracy Analyses

After the exclusion of unassessable stents, accuracy tests with 95% confidence intervals (CIs) comparing 64-MDCT vs invasive coronary angiography for binary ISR detection showed that pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and a diagnostic odds ratio (random effect model) values were 86% (95% CI 80-91%) (Figure 1A), 93% (95% CI 91-95%) (Figure 1B), 12.32 (95% CI 7.26-20.92) (Figure 2A), 0.18 (95% CI 0.12-0.28) (Figure 2B), and 88.9 (95% CI 34.6-228.6), respectively. The area under the sROC curve was 0.94 for per-stent analysis, indicating a high degree of agreement between 64-MDCT and invasive coronary angiography for ISR detection (Figure 3). Visual inspection of Figures 1 and 2 identified the presence of heterogeneity between studies. However, heterogeneity was found statistically significant for specificity (χ2 = 25.81, df = 8, P = .0011), and positive likelihood ratio (χ2 = 24.91, df = 8, P = .002). The calculated positive and negative predictive values were 70.4% and 97.2%, respectively. When including unassessable stents, accuracy tests showed that value of the pooled sensitivity did not change (86%, 95% CI 80-91%), whereas the pooled specificity only slightly decreased (91%, 95% CI 87-92%).

Figure 1

(A) Plot and table of in-stent restenosis detection sensitivity of 64-MDCT vs invasive coronary angiography; (B) plot and table of ISR detection specificity of 64-slice multi-detector computed tomography vs invasive coronary angiography

Figure 2

(A) Plot and table of in-stent restenosis detection negative likelihood ratio of 64-MDCT vs invasive coronary angiography; (B) plot and table of in-stent restenosis detection positive likelihood ratio of 64-slice multi-detector computed tomography vs invasive coronary angiography

Figure 3

Receiver operator characteristic (ROC) curve analysis on a per stented segment basis (nine studies) comparing 64-MDCT vs invasive coronary angiography. The diagnostic accuracy is shown by plotting 1-specificity against sensitivity, area under curve (AUC), and Q* statistic with their standard errors (SE). The upper and lower lines indicate 95% CIs. SROC Summary receiver operating characteristics

Heterogeneity Analyses

The meta-regression analysis to explore the reasons of heterogeneity, including all important covariates, which we presumed to be the most likely source of heterogeneity, did not show any significant effect. Quality assessment for all included studies is shown in Table 2.

Table 2

Quality assessment (QUADAS)

Study	Item 1	Item 2	Item 3	Item 4	Item 5	Item 6	Item 7	Item 8	Item 9	Item 10	Item 11	Item 12	Item 13	Item 14
Gaspar	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Van Mieghem	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Rixe	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Rist	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Ehara	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Oncel	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Cademartiri	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Carrabba	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Manghat	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
All	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes

Item 1: was the spectrum of patients representative of the patients who will receive the test in practice?

Item 2: were selection criteria clearly described?

Item 3: is the reference standard likely to correctly classify the target condition?

Item 4: is the time period between reference and standard and index test short enough to be reasonably sure that the target condition did not change between the two tests?

Item 5: did the whole sample or a random selection of the sample, receive verification using a reference standard of diagnosis?

Item 6: did the patients receive the same reference standard regardless of the index test results?

Item 7: was the reference standard independent of the index test (i.e., the index test did not form part of the reference standard)?

Item 8: was the execution of the index test described in the sufficient detail to permit replication of the test?

Item 9: was the execution of the reference standard described in the sufficient detail to permit its replication?

Item 10: were the index test results interpreted without knowledge of the results of the reference standard?

Item 11: were the reference standard results interpreted without knowledge of the results of the index test?

Item 12: were the same clinical data available when test results were interpreted as would be available when the test is used in practice?

Item 13: were uninterpretable/intermediate test results reported?

Item 14: were withdrawals from the study explained?

Additional Analyses

Results from the bivariate meta-analysis model show negligible differences. Sensitivity, specificity, positive and negative likelihood ratios, and diagnostic odds ratio pooled estimates were 88 (95% CI 80-93%), 94% (95% CI 90-96%), 13.8 (95% CI 8.2-23.2), 0.13 (95% C 0.1-0.2), and 107.20 (95% CI 42.9-267.9), respectively.

Discussion

In this meta-analysis, we focused on the diagnostic performance of the 64-MDCT generation of MDCT as a potential alternative, noninvasive method for ISR detection. We observed an average weighted sensitivity of 86% and specificity of 93% for 64-MDCT in the detection of ISR as compared to invasive coronary angiography. These observations are not substantially different from the meta-analysis by Hamon et al, who included both 16- and 64-MDCT systems.24 While in fact the majority of data were derived from the older 16-MDCT systems, only slightly lower average weighted sensitivity (84%) and specificity (91%) were reported. However, one should realize that both in the meta-analysis of Hamon et al24 as well as the current one, these values were obtained after exclusion of uninterpretable stents. Possibly, the main advantage of improved temporal and spatial resolution may lie in improved rates of interpretable stents. Indeed, with previous generations of MDCT scanners, diagnosis of ISR was particularly limited by the high percentages of stents with insufficient image quality for evaluation. Based on in vitro studies as well as initial clinical investigations, improvement in the rate of interpretable stents has been anticipated with the introduction of 64-MDCT with higher spatial and temporal resolution.15,25 However, thus far no robust data were available to support this expectation. Previous meta-analyses addressing the value of MDCT in the detection of ISR still reported high rates of uninterpretable stents of approximately 13%.24,26 Importantly, these meta-analyses, as described above, included both 16- and 64-MDCT with the majority of data derived from the older 16-MDCT systems. In contrast, the current analysis was restricted to 64-MDCT and showed a relatively slight reduction in the rate of nonassessable stents (9%) as compared to these previous meta-analyses (absolute reduction of 4.4%; relative reduction of 30%). Accordingly, the current analysis supports that the use of 64-MDCT may indeed result in improved stent assessability.

However, despite this improvement as compared to previous investigations, the rate of uninterpretable stents was still not negligible. Moreover, the majority of included studies used strict inclusion criteria with exclusion of patients with smaller stents, contraindication to administration of beta-blocking medication, elevated heart rate and obesity. In previous studies, these factors have been shown to negatively influence stent evaluation with MDCT.7,27,28 Indeed, as can be derived from Table 1, also in the currently included studies, rates of uninterpretable stents tended to be higher in the presence of a higher mean heart rate, no additional administration of beta-blocking medication or a relatively higher proportion of stents with a diameter <3.0 mm. Of note, the results of the present meta-analyses showed significant heterogeneity between the included studies. Thus, our results should be interpreted cautiously, although the use of the random-effects model and the bivariate random-effects meta-analysis model should at least partly correct for this issue. Exploring sources of heterogeneity by meta-regression analyses showed that age, percentage of male, percentage of nonevaluable stents and sample size had no significant influence. Possible sources of heterogeneity were a significant difference in the numbers of assessed stents between studies and the use of kernel convolution filters, which might have influenced the absolute numbers of true and false ISR reported in each study. These variations in-stent analysis may be attributed to the level of experience with 64-MDCT in different study centers. Improvement of technical quality and clinical experience might minimize this difference.

Clinical Implications

The current meta-analysis confirms that in selected patients, 64-MDCT can reliably rule out ISR. The high NPV of 97.2% indicates that if stent patency is demonstrated on 64-MDCT, further evaluation by means of invasive coronary angiography can be avoided. Accordingly, the technique may be a particularly suitable alternative for patients without a high clinical suspicion of ISR and with expected low rate of ISR, but in whom further clinical evaluation is still preferred. However, patients with a high likelihood of ISR, including patients with typical chest pain and/or an unambiguously abnormal stress test, are unlikely to benefit from initial noninvasive evaluation with MDCT. In this patient population, MDCT is limited by its low positive predictive value, inability to determine the hemodynamical relevance of detected lesions and the fact that no immediate intervention can be performed. Accordingly, use of MDCT may only result in unnecessary radiation exposure, although recent developments in technology and software currently allow coronary CT angiography with estimated doses <5 mSv.29,30 In addition, MDCT imaging remains unsuitable for patients with characteristics unfavorable for stent imaging, such as small stents or elevated heart rate. Indeed, it remains important to realize that even with 64-MDCT only in selected patients, the high diagnostic accuracy and relatively low rates of uninterpretable stents are maintained. Routine use of 64-MDCT to rule out ISR in the general population therefore, cannot be recommended. Potentially, improved rates of stent interpretation may be achieved in the near future with the use of dual-source systems with higher temporal resolution,14 high-definition or flat panel detector scanners with improved spatial resolution31,32 or even 320-row systems that allow cardiac imaging in a single volume.33

Conclusions

The results of the current meta-analysis demonstrate that 64-MDCT has a good diagnostic accuracy for ISR detection with a particularly high NPV. However, still a relatively large proportion of stents, although improved as compared to previous reports, remains uninterpretable. Accordingly, only in selected patients, 64-MDCT may serve as a potential alternative noninvasive method to rule out ISR.