The predictive value of the CTA Vasospasm Score on delayed cerebral ischaemia and functional outcome after aneurysmal subarachnoid hemorrhage.

BACKGROUND AND
PURPOSE: Delayed cerebral ischaemia (DCI) is a severe complication of aneurysmal subarachnoid hemorrhage that can significantly impact clinical outcome. Cerebral vasospasm is part of the pathophysiology of DCI and therefore a computed tomography angiography (CTA) Vasospasm Score was developed and an exploration was carried out of whether this score predicts DCI and subsequent poor outcome after aneurysmal subarachnoid hemorrhage.
METHODS: The CTA Vasospasm Score sums the degree of angiographic cerebral vasospasm of 17 intradural arterial segments. The score ranges from 0 to 34 with a higher score reflecting more severe vasospasm. Outcome measures were cerebral infarction due to DCI (CI-DCI), radiological and clinical DCI, and unfavorable functional outcome defined as a modified Rankin Scale >2 at 6 months. Receiver operating characteristic analyses were used to assess predictive value and to determine optimal cut-off scores. Inter-rater reliability was evaluated by Cohen's kappa coefficient.
RESULTS: This study included 59 patients. CI-DCI occurred in eight patients (14%), DCI in 14 patients (24%) and unfavorable outcome in 12 patients (20%). Median CTA Vasospasm Scores were higher in patients with (CI-)DCI and poor outcome. Receiver operating characteristic analysis revealed the highest area under the curve on day 5: CI-DCI 0.89 (95% confidence interval [CI] 0.79-0.99), DCI 0.68 (95% CI 0.50-0.87) and functional outcome 0.74 (95% CI 0.57-0.91). Cohen's kappa between the two raters was moderate to substantial (0.57-0.63).
CONCLUSIONS: This study demonstrates that the CTA Vasospasm Score on day 5 can reliably identify patients with a high risk of developing (CI-)DCI and unfavorable outcome.

INTRODUCTION

Aneurysmal subarachnoid hemorrhage (aSAH) has a high rate of unfavorable outcome, in part related to delayed cerebral ischaemia (DCI) which occurs in about 30% of patients [1]. Cerebral vasospasm (CVS) on angiographic imaging is assumed to play a part in the pathophysiology leading to DCI, which may result in cerebral infarction due to DCI (CI‐DCI) and unfavorable outcome [2]. However, a recent study demonstrated that there is no direct relation between CVS on computed tomography angiography (CTA) and DCI or functional outcome [3]. Studies that investigated the predictive value of CVS on CTA used qualitative measures for assessing the severity of CVS [4]. The aim of this study was to develop a CTA Vasospasm Score to quantify CVS severity and to evaluate whether this score reliably predicts (CI‐)DCI and functional outcome.

METHODS

Patient selection

This study is a post hoc analysis of a prospective diagnostic accuracy study of CTA and transcranial Doppler (TCD) in predicting DCI and functional outcome after aSAH (Transcranial Doppler and CT angiography for investigating cerebral vasospasm in subarachnoid hemorrhage [TACTICS] study). The inclusion of patients with proven aSAH took place within 4 days after onset. In addition, patients had to meet the following inclusion criteria: age 18 years or older and written informed consent from the patient or a legally authorized representative. Moribund patients and patients with contra‐indications for iodine contrast agents were excluded. Baseline characteristics were depicted in the original paper [3]. The original study was registered in the Dutch Trial Register (NTR4157) and the current study was approved by the local research ethics board.

Imaging analysis

Transcranial Doppler and CTA examinations were performed 5 and 10 days after the aSAH. The evaluation of CVS on CTA was independently performed by two neuroradiologists, blinded for clinical information. The degree of CVS of 17 intradural arterial segments was assessed. The CTA Vasospasm Score is the sum of the CVS rating (0 for none, 1 for mild [<50% luminal narrowing] and 2 for severe [>50% luminal narrowing]) in each of the 17 vessels, with a total score ranging from 0 to 34 points. The mean CTA Vasospasm Score of the two raters was used as the final score for statistical analysis. The description of the segments and rating is reported in Appendix 1.

Outcome measures

The primary outcome measure was CI‐DCI on brain magnetic resonance imaging 6 months after aSAH. CI‐DCI was defined as cerebral infarction attributable to DCI after excluding other causes [2]. Secondary outcomes were DCI and functional outcome. Clinical symptoms were assessed during the period of hospital admission and time from ictus to onset of DCI was recorded. DCI was defined by clinical manifestations of DCI or cerebral infarction attributable to DCI on imaging after exclusion of other causes. In comatose patients DCI could only be assessed by imaging (CT or magnetic resonance imaging brain). Functional outcome was assessed with the modified Rankin Scale (mRS) at 6 months, with unfavorable outcome defined as mRS >2 [5].

Statistical analysis

Statistical analysis was performed using IBM SPSS Statistics, version 25. Given the small sample size and non‐Gaussian distribution, medians were compared with the Mann–Whitney U test. Receiver operating characteristic (ROC) analyses were used to determine diagnostic value; the optimal cut‐off value was established with the Youden index and was used to assess diagnostic accuracy with contingency analysis. Cohen's kappa coefficient (κ) was reported for agreement between the two raters. Statistical significance was determined as p < 0.05.

RESULTS

The study included 59 patients. DCI occurred in 14 patients (24%) of whom eight developed CI‐DCI (14%). Twelve of the 59 patients (20%) had an unfavorable outcome. Clinical manifestations of DCI occurred in 12 patients (20%) at a median interval of 4 days (Q1, 1.25; Q3, 9.75). See Appendix 1 for detailed results and analysis.

Median CTA Vasospasm Scores were significantly higher in patients with (CI‐)DCI and unfavorable outcome compared with patients without (CI‐)DCI and favorable outcome, particularly at day 5 (CI‐DCI, p < 0.001; DCI, p = 0.032; unfavorable outcome, p = 0.009). On day 10, the CTA Vasospasm Score was significantly higher in patients with CI‐DCI (p = 0.033) and unfavorable outcome (p = 0.033) but was not significantly higher for patients with DCI (p = 0.069, Figure 1; Appendix 1).

FIGURE 1

Box plots for day 5 and day 10. DCI, delayed cerebral ischaemia; mRS, modified Rankin Scale

Receiver operating characteristic analysis for predicting CI‐DCI revealed an area under the curve (AUC) of 0.89 (95% confidence interval [CI] 0.79–0.99) for the CTA Vasospasm Score with an estimated optimal cut‐off value of 9 at day 5 and an AUC of 0.74 (95% CI 0.55–0.94) for the CTA Vasospasm Score with an estimated optimal cut‐off value of 10 at day 10. ROC analysis for predicting DCI revealed an AUC of 0.68 (95% CI 0.50–0.87) for the CTA Vasospasm Score with an estimated optimal cut‐off value of 8 at day 5 and an AUC of 0.67 (95% CI 0.48–0.86) for the CTA Vasospasm Score with an estimated optimal cut‐off value of 10 at day 10. ROC analysis for predicting functional outcome revealed an AUC of 0.74 (95% CI 0.48–0.89) for the CTA Vasospasm Score with an estimated optimal cut‐off value of 7 at day 5 and an AUC of 0.69 (95% CI 0.48–0.89) for the CTA Vasospasm Score with an estimated optimal cut‐off value of 8 at day 10 (Figure 2; Appendix 1).

FIGURE 2

ROC analysis (sensitivity vs. 1 – specificity). (a) Cerebral infarction due to DCI (CI‐DCI). (b) Delayed cerebral ischaemia (DCI). (c) Unfavorable outcome. ROC, receiver operating characteristic

FIGURE A1

Flowchart of included patients. CI‐DCI, cerebral infarction from DCI; Correctly, correctly categorized by the CTA Vasospasm Score; DCI, delayed cerebral ischaemia; T1, day 5; T2, day 10 [Color figure can be viewed at wileyonlinelibrary.com]

The primary analysis for predicting CI‐DCI had a sensitivity of 1.0, a specificity of 0.69, a positive predictive value (PPV) of 0.33 and a negative predictive value (NPV) of 1.00 at day 5. Sensitivity was also high for the various other outcome measures (range 0.71–0.88), with specificity ranging from 0.47 to 0.62, a PPV ranging from 0.25 to 0.37 and a NPV ranging from 0.87 to 0.97 (Appendix 1).

TABLE A1

Comparison of the CTA Vasospasm Score for outcome

	Day 5	Day 10	Day 5	Day 10
	No CI‐DCI		CI‐DCI
Number of cases	51	51	8	8
Median	6.00	8.50	18.00	15.50
Lower quartile	4.00	3.00	11.50	10.38
Upper quartile	11.00	14.00	26.00	27.13
Minimum	0.00	0.00	10.00	2.00
Maximum	31.50	27.00	27.00	31.00
p value* Mann–Whitney U test	<0.001	0.033

CI‐DCI indicates cerebral infarction due to DCI.

Abbreviations: DCI, delayed cerebral ischaemia; mRS, modified Rankin Scale.

p value Mann–Whitney U test concerns the significance of differences between positive and negative outcome measures noted in the positive column.

Inter‐rater reliability

Inter‐rater reliability was moderate to substantial with a Cohen's kappa of 0.57–0.63 (Appendix 1) (Appendix 2).

TABLE A2

Diagnostic accuracy with contingency analysis

	AUC	(95% CI)	p	J	Cut‐off	Sen	Spe	PPV	NPV	AC	κ rater
CI‐DCI
Day 5	0.89	(0.79–0.99)	0.000*	0.69	9	1.00	0.69	0.33	1.00	0.73	0.63 †
Day 10	0.74	(0.55–0.94)	0.028*	0.46	10	0.88	0.59	0.25	0.97	0.63	0.56 †
DCI
Day 5	0.68	(0.50–0.87)	0.043*	0.34	8	0.71	0.62	0.37	0.88	0.64	0.63 †
Day 10	0.67	(0.48–0.86)	0.055	0.32	10	0.71	0.60	0.36	0.87	0.63	0.56 †
Functional outcome
Day 5	0.74	(0.57–0.91)	0.010*	0.39	7	0.83	0.55	0.31	0.93	0.59	0.66 †
Day 10	0.69	(0.48–0.89)	0.039*	0.34	8	0.83	0.47	0.29	0.92	0.54	0.62 †

Kappa (κ) was used as a measure of agreement between the two raters.

Abbreviations: AC, accuracy; AUC, area under curve; CI, confidence interval; CI‐DCI, cerebral infarction due to DCI; DCI, delayed cerebral ischaemia; J, Youden index; NPV, negative predictive value; PPV, positive predictive value; Sen, sensitivity; Spe, specificity.

Significant p values.

p < 0.000.

DISCUSSION

The CTA Vasospasm Score reliably predicted the occurrence of (CI‐)DCI and poor functional outcome, with the highest accuracy at day 5. Its high sensitivity makes this prediction model suitable for the purpose of screening. CI‐DCI was judged as the most relevant outcome measure since it is less affected by other deterioration causes. This finding is in accordance with the recommendation of an international expert panel [2].

Several publications looking at the prediction of (CI‐)DCI and functional outcome with various models based on patient characteristics and diagnostic modalities have been published [4]. All previous models have their advantages and disadvantages but comparing results should be done cautiously. A recent machine learning (ML) study, using patient characteristics on day 1–3 after aSAH, showed an AUC of 0.75 (95% CI 0.64–0.84) for predicting DCI and an AUC of 0.89 (95% CI 0.81–0.94) for functional outcome at 3 months. This model performed better than previous regression models [6]. Compared to our CTA Vasospasm Score, the ML model has a slightly higher AUC predicting DCI and functional outcome. However, it did not report CI‐DCI, our primary outcome, with the highest AUC of 0.89 (95% CI 0.79–0.99). Since the ML model used 31 variables for predicting DCI, the causal relation and mutual coherence remain uncertain [6]. Its advantage is that no additional diagnostics are required, but it is less suitable for follow‐up given the many variables. Therefore, the ML model is less suitable to decide on interventions and treatments to achieve a better outcome at the bedside. Furthermore, our method showed a clear relationship between CTA Vasospasm Score and outcome, on the basis of which treatment may be initiated and monitored. In addition, the high AUC values showed that the CTA Vasospasm Score could contribute as a variable in future multimodal (regression) analyses or ML models. Another digital subtraction angiography (DSA) study used a similar grading system as in our study [7], but unfortunately an accuracy analysis was not performed. Although DSA is the gold standard in CVS diagnostics, its invasive nature makes DSA less appropriate as a screening method. Nevertheless, it emphasizes that CVS correlates with unfavorable outcome, in accordance with our findings.

Digital subtraction angiography, CTA and CT perfusion perform similarly in predicting DCI. In several studies, sensitivity and specificity were found to be between 0.70 and 0.80 [4]. In TACTICS, a comparable sensitivity of 0.81 but with a specificity of 0.070 was found. TCD had a sensitivity of 0.44 and a specificity of 0.67. However, our analysis of both CTA and TCD did not accurately predict DCI or functional outcome [3].

Compared to other imaging studies, the sensitivity achieved in the current study up to 1.0 is good, with a specificity up to 0.69 comparable to other studies. This underscores the added value of the CTA Vasospasm Score.

Our study also has limitations. First, (CI‐)DCI and unfavorable outcome had low incidence, with a low PPV. Secondly, given the relatively small sample size, confidence intervals of significant findings are wide. Thirdly, our cut‐off values were optimized for the study data. In practice, one cut‐off value for different situations would be useful. As such, our results indicate a useful method but require confirmation in a larger prospectively collected sample of aSAH patients.

CONCLUSIONS

Based on our findings, the CTA Vasospasm Score at day 5 with estimated cut‐off values can reliably identify patients with a high risk of developing CI‐DCI at 6 months after aSAH.

CONFLICT OF INTEREST

None.