Collateral Status in Ischemic Stroke: A Comparison of Computed Tomography Angiography, Computed Tomography Perfusion, and Digital Subtraction Angiography.

OBJECTIVE: To compare assessment of collaterals by single-phase computed tomography (CT) angiography (CTA) and CT perfusion-derived 3-phase CTA, multiphase CTA and temporal maximum-intensity projection (tMIP) images to digital subtraction angiography (DSA), and relate collateral assessments to clinical outcome in patients with acute ischemic stroke.
METHODS: Consecutive acute ischemic stroke patients who underwent CT perfusion, CTA, and DSA before thrombectomy with occlusion of the internal carotid artery, the M1 or the M2 segments were included. Two observers assessed all CT images and one separate observer assessed DSA (reference standard) with static and dynamic (modified American Society of Interventional and Therapeutic Neuroradiology) collateral grading methods. Interobserver agreement and concordance were quantified with Cohen-weighted κ and concordance correlation coefficient, respectively. Imaging assessments were related to clinical outcome (modified Rankin Scale, ≤ 2).
RESULTS: Interobserver agreement (n = 101) was 0.46 (tMIP), 0.58 (3-phase CTA), 0.67 (multiphase CTA), and 0.69 (single-phase CTA) for static assessments and 0.52 (3-phase CTA) and 0.54 (multiphase CTA) for dynamic assessments. Concordance correlation coefficient (n = 80) was 0.08 (3-phase CTA), 0.09 (single-phase CTA), and 0.23 (multiphase CTA) for static assessments and 0.10 (3-phase CTA) and 0.27 (multiphase CTA) for dynamic assessments. Higher static collateral scores on multiphase CTA (odds ratio [OR], 1.7; 95% confidence interval [CI], 1.1-2.7) and tMIP images (OR, 2.0; 95% CI, 1.1-3.4) were associated with modified Rankin Scale of 2 or less as were higher modified American Society of Interventional and Therapeutic Neuroradiology scores on 3-phase CTA (OR, 1.5; 95% CI, 1.1-2.2) and multiphase CTA (OR, 1.7; 95% CI, 1.1-2.6).
CONCLUSIONS: Concordance between assessments on CT and DSA was poor. Collateral status evaluated on 3-phase CTA and multiphase CTA, but not on DSA, was associated with clinical outcome.

The clinical outcome of acute ischemic stroke (AIS) patients is associated with the grade of collateral filling.[1] It is hypothesized that the ischemic process can be slowed down by collateral vessels perfusing the tissue that is still salvageable, which is termed the penumbra.[2] In patients who undergo endovascular treatment (EVT), good collateral filling is associated with smaller final infarct volumes and better clinical outcomes than when the collateral filling is poor or absent.[2-4]

In the past decade, collateral evaluation has gained much interest and multiple grading systems for different imaging modalities have been developed.[5,6] Computed tomography angiography is widely used for the evaluation of patients with suspected AIS. It is a fast, relatively inexpensive and noninvasive method for assessing occlusions and potential causes of stroke. A single time frame snapshot (single-phase CTA) of the collateral status can be assessed and used for treatment guidance and predicting patient outcomes. However, a single time frame snapshot may make it difficult to grade collateral circulation in case of delayed contrast arrival.[2,7] Multiphase CTA may solve the issues that come with single-phase CTA.[8,9] In addition, multiphase CTA has proven to be superior to grading collaterals on single-phase CTA in terms of interobserver agreement, predicting final infarct core volume and predicting functional outcome.[8,9] Patients with suspected stroke often undergo both CTA and CT perfusions (CTP). Computed tomography perfusion is the ultimate multiphase CTA study since it typically includes 40 to 50 time frames obtained at a 1- to 3-second temporal resolution and monitors the contrast agent from wash-in to wash-out. Therefore, CTP source images can be used to reconstruct 3-phase CTA, multiphase CTA and temporal maximum-intensity projection (tMIP) images. Digital subtraction angiography (DSA) is a 2D study with high spatial and temporal resolution and high contrast making it the ultimate dynamic study of the cerebral vessels and the reference standard for assessing collateral circulation in AIS patients.[10] However, DSA is invasive, relatively costly and time consuming, which is undesirable in the acute stroke setting. Digital subtraction angiography is, therefore, only routinely performed in patients who subsequently undergo EVT.

With regard to collateral assessments, single-phase CTA, CTP-derived 3-phase CTA, multiphase CTA, tMIP images, and DSA have not been compared before.

AIMS

In this study, we aimed to compare assessment of collateral filling on single-phase CTA and CTP-derived 3-phase CTA, multiphase CTA, and tMIP images in terms of interrater reliability and agreement between imaging modalities. In a secondary analysis, we aimed to evaluate the relation between collateral assessments and 90-day clinical outcome.

MATERIALS AND METHODS

Patient Selection

Consecutive adult patients with AIS from Stanford Medical Center, who were considered for EVT between 2010 and 2018, were selected for this study. Inclusion criteria were (1) presence of CTP and CTA images that were acquired as part of the acute stroke protocol and (2) occlusion of the internal carotid artery (ICA) and/or occlusion of the M1 and/or M2 segment of the middle cerebral artery. Patients were excluded in case of bilateral stroke. Digital subtraction angiography cases were excluded if the filming did not extend into the late venous phase, patient motion precluded adequate interpretation, or if only anteroposterior images were available for analysis. The need for informed consent was waived by the local institutional research board for this retrospective analysis of data, which was collected as part of clinical practice. Of the 530 patients who were considered for EVT 101 were included for the interobserver analysis. Eighty patients had interpretable DSA and 74 had clinical outcome assessed at 90 days (Fig. 1).

FIGURE 1

Flowchart of patient selection and analysis steps. M1, M1 segment of middle cerebral artery; M2, M2 segment of middle cerebral artery.

Patient Characteristics

Baseline data included demographics and cardiovascular risk factors, such as hypertension, diabetes mellitus, hyperlipidemia, smoking, atrial fibrillation, prior cerebrovascular accident, prior coronary artery disease, and use of antiplatelet medication or anticoagulants. Furthermore, we collected admission National Institutes of Health Stroke Scale and data on treatment including administration of intravenous tissue-type plasminogen activator, EVT, and whether reperfusion was achieved, defined as a Thrombolysis in cerebral infarction Scale of IIB or III. We also recorded time from symptom onset to treatment.

Image Acquisition

Different CT scanners were used from Siemens Healthcare (Erlangen, Germany) and General Electric Healthcare (Milwaukee, WI). Computed tomography perfusion was performed with 80 kV and 100 mAs. Thirty-seven phases at 1-second time interval followed by 33 phases at 3-second time interval were acquired. Either one run with 10-mm slices or 2 runs with 5-mm slices were performed covering at least ASPECTS levels 1 and 2 of the brain.

The CTA studies of the carotid arteries were obtained on 64-slice CT scanners. The image acquisition protocol was as follows: spiral mode, a 0.6- to.8-second gantry rotation; collimation, 64 × 0.5–1 mm; pitch, around 1; slice thickness, 1 to 1.25 mm; reconstruction interval, 0.75 to 0.1 mm, and acquisition parameters, 120 kVp/240 mA. A caudocranial scanning direction was selected, covering the midchest to the vertex of the brain. Seventy to 80 mL of Isovue 300 or 370 (lopamidol; Bracco Diagnostics Inc, Monroe Township, NJ) was injected into an antecubital vein with a power injector at a rate of 4 to 5 mL per second. Optimal timing of the CTA acquisition was achieved using a test bolus technique.

A Siemens Artis Zee biplane neuroangiography unit was used for EVT. The DSA images were acquired during EVT with standard anteroposterior and lateral views following cervical ICA contrast injection of the afflicted side.

Image Preparation

Single-phase CTA images were created from thin-slice data and saved as 20-mm maximum intensity projection (MIP) images. The images were captured on ASPECTS levels 1 (basal ganglia) and 2 (at the top of the lateral ventricles), respectively.[11]

The CTP source data were used to create 3-phase CTA, multiphase CTA and tMIP images. Three-phase CTA images were prepared using the CTP images with the thinnest slice thickness available (ranging from 1.5 to 10 mm). The first phase was selected based on the peak of arterial inflow, which was automatically calculated and checked visually. The second phase was timed 8 seconds after the first phase and the third phase was timed 8 seconds after the second phase.[8] Similar to single-phase CTA, 20-mm MIP images were captured on ASPECTS levels 1 and 2, respectively.

Multiphase CTA was created from all the CTP time-phases containing thick slice images (ranging from 5 to 10 mm) on the 2 ASPECTS levels. To assess the collateral score dynamically, the series were saved as video files.

All the CTP time-phases were reconstructed into tMIP images, where the highest Hounsfield unit was taken for every voxel with perfusion analysis software (Intellispace Portal, version 6.0; Philips Healthcare, Best, the Netherlands). The MIP thickness was set to 20 mm and the images were captured on ASPECTS levels 1 and 2. The reconstructions were created automatically by the software.

Regarding the reference standard, all the available DSA images, which were acquired after CT, were used for determining the grade of collateral circulation before EVT took place.

Interobserver Study

The single-phase CTA, 3-phase CTA, multiphase CTA, and tMIP images were presented separately to 2 separate neuroradiologists (J.D.—19 years of experience and M.W.—21 years of experience). The images were anonymized and placed in a randomized order for the reviews. The observers were blinded to clinical information and other imaging data except for the occlusion site. On single-phase CTA, 3-phase CTA, multiphase CTA, and tMIP images, the collaterals in the affected territory were graded using a static collateral score: 0, absent collaterals; 1, collaterals filling 50% or less of the occluded territory; 2, collaterals filling greater than 50%, but less than 100% of the occluded territory; 3, collaterals filling 100% of the occluded territory.[12-14] For 3-phase CTA, multiphase CTA and DSA. The modified American Society of Interventional and Therapeutic Neuroradiology (ASITN) score[15] was used, which enables dynamic grading of collaterals: 0, nonexistent or barely visible pial collaterals on the ischemic site during any point of time; 1, partial collateralization of the ischemic site until the late venous phase; 2, partial collateralization of the ischemic site before the venous phase; 3, complete collateralization of the ischemic site by the late venous phase; 4, complete collateralization of the ischemic site before the venous phase.[15,16] Collateral assessment on DSA images was done by a third observer (J.H.—6 years of experience) and was used as the reference standard. The decision for choosing one DSA observer was based on previous studies.[17,18] No consensus meetings were arranged.

Clinical Outcome

Favorable clinical outcome was defined as a modified Rankin Scale (mRS) 0–2, 90 days after the index event.

Statistical Analysis

We reported frequencies and percentages for categorical variables and medians with first (Q1) and third (Q3) quartiles for continuous variables. Since collateral scores are an ordinal variable, the agreement between the 2 observers was quantified by calculating Cohen κ, with disagreements weighted according to their squared distance from perfect agreement.[19] The level of agreement was categorized and based on the κ values: poor, <0.20; fair, 0.21 to 0.40; moderate, 0.41 to 0.60; good, 0.61 to 0.80; very good, 0.81 to 1.00. Confidence intervals for κ statistics were based on 5000 bootstrap resamples. Concordance between the pooled CT measurements and DSA measurements was quantified using percentages of agreement and concordance correlation coefficients (CCCs).[20,21] Because clinical decision making is often based on the dichotomized (either poor or good collateral supply) collateral score, we also evaluated the dichotomized (0–1 vs 2–3) static collateral score and modified (m)ASITN (0–2 vs 3–4) score with respect to the comparison between CT and DSA.[13,22,23] Lastly, we quantified the relation between the imaging assessments on the original and binary scales and the primary outcome (90-day mRS, ≤ 2) by using binary logistic regression, and we adjusted for recanalization status.[24] Association measures were reported as odds ratios (OR) and 95% confidence intervals (CI). The statistical analyses were performed in R (version 3.5.0).

RESULTS

Patients with (n = 80) and without (n = 21) interpretable DSA acquisitions were compared (Table 1). Median age of the total group (n = 101) was 75 (Q1–Q3, 67–82), and 38 (38%) patients were men.

TABLE 1

Patient Characteristics, Overall and Stratified by Availability of DSA

Characteristics	Total (N = 101)	DSA (n = 80)	No DSA (n = 21)
Age: median (Q1–Q3), y	75 (67–82)	74 (67–83)	76 (69–81)
Male sex	38 (38)	28 (35)	10 (48)
Admission NIHSS, median (Q1–Q3)	15 (11–20)	14 (11–20)	20 (14–23)
Time from symptom onset to CTP	127 (62–332)	115 (64–324)	195 (58–411)
Intravenous tPA	52 (52)	40 (50)	12 (57)
Time to tPA (min.), median (Q1–Q3)	87.0 (56–151)	79.0 (55–155)	98 (60–125)
EVT	91 (90)	76 (95)	15 (71)
Reperfusion (TICI IIB-III)	71 (78)	59 (78)	12 (80)
Medical history, n (%)
Hypertension	81 (80)	64 (80)	17 (81)
Diabetes mellitus	22 (22)	14 (18)	8 (40)
Hyperlipidemia	58 (59)	47 (60)	11 (55)
Smoking status
Current	15 (16)	13 (17)	2 (11)
Former	28 (30)	23 (30)	5 (26)
Never	52 (55)	40 (53)	12 (63)
Atrial fibrillation	54 (54)	40 (50)	14 (67)
Prior antiplatelet or anticoagulant therapy	56 (56)	47 (59)	9 (45)
Prior CVA	22 (22)	17 (21)	5 (25)
Prior CAD	30 (30)	24 (30)	6 (30)
Imaging findings, n (%)
Left side occluded on CTA	61 (60)	46 (58)	15 (71)
Occlusion site on CTA
ICA	18 (18)	7 (9)	11 (52)
M1	54 (54)	50 (63)	4 (19)
M2	22 (22)	19 (24)	3 (14)
Multiple sites	7 (7)	4 (5)	3 (14)
Follow-up (n = 76), n (%)
Favorable clinical outcome at 90 days*	47 (62)	38 (59)	9 (75)

Categorical characteristics are summarized as counts and percentages (calculated out of nonmissing values), and continuous characteristics are summarized as medians with first (Q1) and third (Q3) quartiles.

* Defined as mRS ≤ 2.

NIHSS indicates National Institutes of Health Stroke Scale; TICI, thrombolysis in cerebral infarction; CVA, cerebrovascular accident; CAD, coronary artery disease; M1, M1 segment of middle cerebral artery; M2, M2 segment of middle cerebral artery.

Observed static collateral scores (Supplemental Table I, http://links.lww.com/RCT/A105) were generally high in this population: the number of observations ranged from 0 to 3 (0–3%) for collateral score 0, from 8 to 23 (8–23%) for collateral score 1, from 22 to 44 (22–44%) for collateral score 2 and from 42 to 68 (42–68%) for collateral score 3. Similarly, the number of mASITN scores ranged from 1 to 2 (1–2%) for mASITN 1, from 4 to 19 (4–19%) for mASITN 2, from 17 to 45 (17–45%) for mASITN 3, and from 7 to 47 (7–47%) for mASITN 4.

Disagreements between observers did not differ more than 1 point on the collateral scales, resulting in disagreements between the dichotomized scales in 10 cases.

CTA and DSA images from an example case with good collateral circulation are shown in Figure 2 and Figure 3, respectively. The CTA and DSA images from an example case with poor collateral circulation are shown in Figure 4 and Figure 5, respectively.

FIGURE 2

Example case with collateral circulation graded as good by the 2 observers. Single-phase CTA images are shown on ASPECTS levels 1 (A) and 2 (B). Three-phase CTA images were created from PCT source data. The first phase of the 3-phase CTA images is shown on ASPECTS levels 1 (C) and 2 (D).

FIGURE 3

Example case identical to the case in Figure 2. Anteroposterior (A) and lateral (B) digital subtraction angiography views in the early contrast phase are shown. Anteroposterior (C) and lateral (D) views of the late contrast phase show good filling of the collateral circulation in the middle cerebral artery territory (white dotted shapes).

FIGURE 4

Example case with collateral circulation graded as poor by the 2 observers. Single-phase CTA images are shown on ASPECTS levels 1 (A) and 2 (B). Three-phase CTA images were created from PCT source data. The first phase of the 3-phase CTA images is shown on ASPECTS levels 1 (C) and 2 (D).

FIGURE 5

Example case identical to the case in Figure 4. Anteroposterior (A) and lateral (B) digital subtraction angiography views in the early contrast phase are shown. Anteroposterior (C) and lateral (D) views of the late contrast phase show poor filling of the collateral circulation in the middle cerebral artery territory (white dotted shapes).

Interobserver agreement was 0.46 (tMIP), 0.58 (3-phase CTA), 0.67 (multiphase CTA), and 0.69 (single-phase CTA) for static assessments and 0.52 (3-phase CTA) and 0.54 (multiphase CTA) for dynamic assessments (Fig. 2).

Concordance between the CT observations and the reference standard (DSA) is summarized in Table 2 and displayed in Figure 6. Agreement (range, 29–53%) and concordance (CCC range, 0.08–0.27) on the ordinal scale were poor. Agreement improved after dichotomization of the collateral scores (range, 54–81%), but concordance remained poor (CCC range, −0.02 to 0.24).

TABLE 2

Measures of Concordance Between Collateral Scores Assessed on CT and DSA (Reference Standard)

Grading Method	Ordinal Score		Dichotomized Score
	Agreement (%)	CCC (95% CI)	Agreement (%)	CCC (95% CI)
CS single-phase	53	0.09 (−0.06–0.24)	78	0.02 (−0.13–0.17)
CS 3-phase	48	0.08 (−0.06–0.22)	81	−0.11 (−0.26–0.05)
CS multiphase	41	0.23 (0.08–0.37)	78	0.01 (−0.14–0.15)
CS tMIP	29	0.19 (0.05–0.32)	82	−0.02 (−0.18–0.13)
mASITN 3-phase	29	0.10 (−0.02–0.21)	54	0.12 (−0.01–0.25)
mASITN multiphase	40	0.27 (0.12–0.40)	60	0.24 (0.09–0.38)

CS indicates collateral score;

FIGURE 6

Estimated CCCs and weighted κ statistics with 95% CIs. Confidence intervals for κ statistics were based on 5000 bootstrap resamples. CS indicates collateral score.

Clinical outcome was successfully collected in 76 (75%) patients. Patient characteristics did not differ significantly between patients who had follow-up and patients who did not have follow-up. Adjusted for recanalization, higher static collateral scores on multiphase CTA (OR, 1.7; 95% CI, 1.1–2.7) and tMIP (OR, 2.0; 95% CI, 1.1–3.4) were associated with favorable clinical outcome (Table 3). Similarly, higher mASITN scores on 3-phase CTA (OR, 1.5; 95% CI, 1.1–2.2) and multiphase CTA (OR, 1.7; 95% CI, 1.1–2.6) were associated with favorable clinical outcome. For single-phase CTA (OR, 1.1; 95% CI, 0.7–1.8) and 3-phase CTA (OR, 1.6; 95% CI, 1.0–2.6), a positive trend was observed for the association with favorable clinical outcome. No significant associations were found between DSA assessments and clinical outcome.

TABLE 3

Associations Between Imaging Assessments and 90-Day Clinical Outcome (Modified Rankin Scale <3)

	Crude OR (95% CI)	Adjusted* OR (95% CI)
Collateral score
Single-phase	1.1 (0.7–1.7)	1.1 (0.7–1.8)
3-phase	1.6 (1.0–2.6)	1.6 (1.0–2.6)
Multiphase	1.5 (0.9–2.2)	1.7 (1.1–2.7)†
tMIP	2.0 (1.1–3.5)†	2.0 (1.1–3.4)†
mASITN
3 Phase	1.5 (1.1–2.1)†	1.5 (1.1–2.2)†
Multiphase	1.5 (1.0–2.1)†	1.7 (1.2–2.6)†
Collateral score DSA
Original scale	0.7 (0.4–1.4)	0.8 (0.4–1.6)
Binary scale	0.4 (0.1–1.7)	0.4 (0.1–1.9)
mASITN DSA
Original scale	0.8 (0.5–1.4)	0.9 (0.5–1.6)
Binary scale	1.2 (0.4–3.3)	1.5 (0.5–4.7)

* Adjusted for recanalization status.

† P < 0.05.

DISCUSSION

We compared single-phase CTA and CTP-derived 3-phase CTA, multiphase CTA and tMIP with DSA with respect to their ability to reliably grade collateral circulation in AIS patients. We evaluated 2 scoring systems: a static collateral score entailing 4 categories and a dynamic collateral score (mASITN) entailing 5 categories. Agreement between the 2 observers was moderate to good. Collateral assessment achieved similar interobserver agreement for all imaging modalities and concordance with DSA was in general poor. Associations with favorable outcome were significant for CTP-derived 3-phase CTA, multiphase CTA and tMIP assessments, but not for single-phase CTA or DSA assessments.

The observed interobserver agreement in this study was comparable to previously reported measures of agreement. Previous studies found κ values for single-phase CTA of 0.49,[13] 0.87[14] and 0.68,[17] which is consistent with the results of our study (κ, 0.69). Multiphase CTA has been proposed as a reliable tool for assessing collateral grade in patients with AIS.[8,9] Interobserver agreement was found to be very good (κ, 0.81) in one small study, while we found a κ of 0.58.[8]

The overall concordance between CT assessments and DSA was poor, despite the reasonable agreement between the observations on CT and DSA. Studies evaluating the concordance between CT and DSA are scarce. One study found poor agreement between single-phase CTA and DSA (κ, 0.24), which is in line with the observations in this study.[17] Another study found a modest correlation between anterior circulation collaterals on CTA and DSA, but different collateral scores were used for each imaging modality.[18] Another study found good agreement (82%) and a correlation (Spearman correlation coefficient 0.83) between multiphase CTA and DSA.[25] Three-point collateral scales were used, but no concordance measures were given. As a result, the percentage of agreement corresponds with our study, but we were not able to compare concordance measures between multiphase CTA and DSA. An explanation for the observed concordance may be the fact that neurointerventionalists do not usually perform DSA of either the contralateral ICA or the posterior circulation in anterior circulation stroke. As a result, the DSA assessment of collateral filling is likely to be incomplete relative to information from CTA and CTP. Also, the collateral scales on each modality and technique may be sufficiently different that they do not correlate. Computed tomography (CT) angiography lacks the temporal resolution of DSA, and CTA scales cannot capture the time resolved blood flow into the collateral circulation. For instance, if one looks at time point 1 and time point 2, which are separated by 20 seconds on a multiphase CTA, the collateral score might be good on the CTA scale. However, the DSA scale might show that it takes a very long time for the collateral vessels to fill, which might translate to a poor collateral score. Perfusion maps generated from CTP may offer more information than CTA because they allow evaluation of blood flow on a tissue level. However, evaluation of perfusion maps was beyond the scope of this study.

Collateral grading has been established as a predictor of clinical outcome independent of the number of phases acquired with CT.[4,26-28] In our study, single-phase assessments were not significantly related to favorable clinical outcome, although a positive trend was observed: the more acquired CT phases, the stronger the associations between the collateral grading and clinical outcome. This is in line with a previous study, which reported that the association with clinical outcome was stronger for multiphase assessments than for single-phase assessments.[9] Similarly, one study found that time-invariant CTA was superior to single-phase CTA in terms of predicting clinical outcome with the collateral score.[7] These results confirm the added value of multiple-phase CTA, either created with CTA or with CTP, when it comes to prediction of clinical outcomes.

The association between DSA assessments and clinical outcome has been investigated in one study, but no association was observed.[17] This is in line with our study in which we did not observe an association between DSA assessments and clinical outcome. However, the results from this particular analysis must be interpreted with caution because selection bias cannot be ruled out completely. Still, the selection process in this study reflects routine clinical care and therefore we believe the risk of selection bias is low. Future studies could elucidate whether prediction of clinical outcome is indeed more feasible with CTA than with DSA.

We calculated measures of concordance (CCCs) enabling us to evaluate accuracy and consistency between the CT assessments from 2 observers and the DSA-assessments from one observer. However, CCC and other measures of agreement are sensitive to imbalanced data as was the case in our study. To solve this issue, a case control design may be used, but we did not have enough cases to use such a design. Patients with poor collaterals are unlikely to be selected for EVT, because they likely had larger infarctions that would exclude them from thrombectomy eligibility. As a result, the predominant number of patients with good collateral supply in our study leads to an increased risk of agreement by chance only, resulting in poor CCCs. In our clinic, all patients with suspected AIS either undergo CT or MRI or both. Selection bias may have been an issue as many patients were excluded from this study because no CT data were available. The majority of these patients was imaged with MRI instead of CT. Evaluation of collateral grading on MRI was beyond the scope of this study. We do not believe that collateral filling is associated with the choice of imaging modality, because it is expected that patients with poor collateral filling preferably undergo CT as their clinical condition is probably worse than patients with good collateral filling. Moreover, poor agreement between assessments on CTA and DSA was also observed in a more balanced population before.[17]

Still, the imbalanced distribution of collateral filling is a limitation of our study, and future studies could, therefore, reassess measures of correlation between CT and DSA in more balanced populations. Patients who did not undergo DSA after all were excluded from the concordance analyses. We do not believe that this elevates the risk of selection bias as the selection process in this study reflects routine clinical care. Another limitation of our study is that we captured and assessed images on the 2 ASPECTS levels only. This was done to achieve a standardized assessment. This could have influenced the collateral assessments as the observers were not able to look at the whole vasculature of the brain. Lastly, we did not evaluate commercially available CT grading software for assessing collateral filling in patients with AIS. In our opinion the method of assessment of collateral filling should also be possible without specialized software since it is not standardly available. Future studies could, however, evaluate how collateral grading done by observers compares to automated grading tools.

In conclusion, collateral assessment achieved similar interobserver agreement for all imaging modalities and concordance with DSA was in general poor. Collateral status evaluated on multiphase CTA was associated with clinical outcome, which was not the case for DSA.