Computed tomography angiography-based deep learning method for treatment selection and infarct volume prediction in anterior cerebral circulation large vessel occlusion.

BACKGROUND: Computed tomography perfusion (CTP) is the mainstay to determine possible eligibility for endovascular thrombectomy (EVT), but there is still a need for alternative methods in patient triage.
PURPOSE: To study the ability of a computed tomography angiography (CTA)-based convolutional neural network (CNN) method in predicting final infarct volume in patients with large vessel occlusion successfully treated with endovascular therapy.
MATERIALS AND METHODS: The accuracy of the CTA source image-based CNN in final infarct volume prediction was evaluated against follow-up CT or MR imaging in 89 patients with anterior circulation ischemic stroke successfully treated with EVT as defined by Thrombolysis in Cerebral Infarction category 2b or 3 using Pearson correlation coefficients and intraclass correlation coefficients. Convolutional neural network performance was also compared to a commercially available CTP-based software (RAPID, iSchemaView).
RESULTS: A correlation with final infarct volumes was found for both CNN and CTP-RAPID in patients presenting 6-24 h from symptom onset or last known well, with r = 0.67 (p < 0.001) and r = 0.82 (p < 0.001), respectively. Correlations with final infarct volumes in the early time window (0-6 h) were r = 0.43 (p = 0.002) for the CNN and r = 0.58 (p < 0.001) for CTP-RAPID. Compared to CTP-RAPID predictions, CNN estimated eligibility for thrombectomy according to ischemic core size in the late time window with a sensitivity of 0.38 and specificity of 0.89.
CONCLUSION: A CTA-based CNN method had moderate correlation with final infarct volumes in the late time window in patients successfully treated with EVT.

Introduction

Endovascular thrombectomy (EVT) has been the standard of care for some years now for patients with ischemic stroke and large vessel occlusion (LVO) presenting within 6 h of symptom onset. With large trials showing the efficacy and safety of endovascular therapy up to 24 h after time from last known well, the need for advanced neuroimaging and interpretation of these studies has surged.

The main questions to answer for determining eligibility for thrombectomy are: (1) is there salvageable brain tissue, that is, is there a mismatch between the ischemic core and penumbra and (2) how large is the ischemic core. Currently, both computed tomography perfusion (CTP)-based methods and magnetic resonance imaging (MRI)-based perfusion-weighted imaging with diffusion-weighted imaging (DWI) are the mainstay for determining the core and penumbra in stroke diagnosis and treatment selection. In patients with stroke symptoms, computed tomography angiography (CTA) is routinely acquired for LVO detection and detecting significant carotid artery stenoses.

With the growing number of patients possibly eligible for thrombectomy, the need for patient triage has surged, and there is still a need for alternative triage methods, as current imaging methods do not always provide the information needed or they may not be available in hospitals outside of comprehensive stroke centers.

To this end, several studies have looked at CTA-based deep learning methods in ischemic stroke detection with promising results[4-7] and our previous study suggested that a convolutional neural network (CNN) model could be useful in determining eligibility for thrombolytic therapy. However, CNN performance in final infarct estimation might vary depending on treatment selection and could be different in patients receiving thrombolytic therapy versus patients treated with EVT.

In this study, we set out to investigate whether our CTA-based CNN model can predict the final infarct volume in patients with acute ischemic stroke (AIS) treated successfully with mechanical thrombectomy. The premise was that with successful recanalization, the baseline infarct core determined by the CNN should roughly match the final infarct volume on follow-up imaging. CNN performance was analyzed for patients presenting either in the early (0–6 h) or late (6–24 h) time window and compared to a commercially available CTP-based software (RAPID, iSchemaView) to investigate the ability of this method in predicting final infarct volume.

Materials and methods

The data that support the findings of this study are available from the corresponding author upon reasonable request. Helsinki University Hospital ethics committee approved this retrospective study and patients’ informed consent was waived.

Study population

We retrospectively studied the clinical and imaging findings of consecutive stroke suspected cases that presented to Helsinki University Hospital between February 2018 and March 2020. The total number of thrombectomies performed in our institution during this time period was 505. Inclusion criteria for this study were: (1) stroke code activated, (2) admission stroke protocol imaging performed using fast CTA acquisition protocol and CTP, (3) anterior circulation LVO at CTA (CCA, ICA, MCA M1, MCA M2), (4) successful EVT, defined as Thrombolysis in cerebral infarction score 2b or 3 and (5) a follow-up NCCT or an MRI study with DWI performed no later than 5 days after the onset of symptoms. Patients with hemorrhagic transformation of infarct resulting in parenchymal hematoma with mass effect were excluded. This exclusion was done so that the hematomas and related edema would not produce errors in infarct volume measurements. Patient characteristics are presented in Table 1.

Table 1.

Patient characteristics.

No. of patients	89
Age, mean (SD, range)	67 (13.3, 28-92)
Female, n (%)	46 (52)
NIHSS, median (IQR) a	12 (7–17)
Time from symptom onset to CT imaging (min), median (IQR) b	121 (71–228)
Time from symptom onset to recanalization (min), median (IQR)2	215 (169–348)
Most proximal target occlusion location, n (%)
CCA	1 (<1)
Proximal ICA	3 (3)
Distal ICA	10 (11)
MCA M1	54 (61)
MCA M2	21 (24)
Intravenous thrombolysis, n (%)	40 (45)

aNIHSS was reported for 86 patients.

bExact time from symptom onset was unknown in 32 patients.

SD: standard deviation; IQR: interquartile range; NIHSS: national institutes of health stroke scale; CT: computed tomography; CCA: common carotid artery; ICA: internal carotid artery; MCA: middle cerebral artery; CNN: convolutional neural network; CTP-RAPID: computed tomography perfusion RAPID; CBF: cerebral blood flow.

Image acquisition and preprocessing

A majority of patients (n = 82) were imaged in the acute setting using a Siemens Somatom Definition Edge (Siemens Healthineers, Erlangen, Germany) 128-slice CT scanner. The CTA imaging parameters were tube voltage 120 kVp, reference current time 150 mAs, pitch 1.3, reconstruction kernel I30f, and slice thickness/increment 0.75/0.5 mm. The iodine concentration of the contrast agent was 350 mg/mL with an amount of 50 mL and injection rate of 5 mL/s. The timing of the scan was 12 s after time to peak of the test bolus in the ascending aorta. Five patients were imaged with a 128-slice Siemens Somatom Definition Flash (Siemens Healthineers, Erlangen, Germany) and two patients with a 128-slice GE Revolution EVO (GE Healthcare, Milwaukee, WI). CT was used for follow-up in 78 patients and 11 patients had a follow-up MRI. All follow-up studies were performed in our institution and a majority (n = 64) were performed with the same scanner as the CTA (Siemens Somatom Definition Edge). Six follow-up CT studies were performed with a Siemens Somatom Definition Flash and eight with a GE Revolution EVO. Follow-up MRI studies were performed with a Siemens Magnetom Verio 3T (n = 5), a Siemens Magnetom Skyra 3T (n = 3), and a Siemens Magnetom Avanto 1.5 T (n = 3). Almost all follow-up studies (n = 87) were performed 24 h after admission. In two patients the follow-up study was performed 2 or 3 days from admission. Images were anonymized and stored on a server running the Extensible Neuroimaging Archive Toolkit, or XNAT, version 1.1.6.

All follow-up studies were evaluated for final infarct volume. A senior neuroradiologist (MK) and a radiologist in training (LH), with over 20 and over 5 years of experience, respectively, segmented the infarcted regions on follow-up CT and diffusion weighted MRI scans in consensus using 3D Slicer image processing and visualization platform. No blinding was used regarding image assessment. Image data preprocessing and 3D convolutional neural network implementation was conducted by a physicist (TM).

The CNN had been previously trained and validated and has been used in a recent publication. A total of 150 patients with a suspected AIS of the middle cerebral artery territory were retrospectively selected for CNN development. From this population, 75 were diagnosed with stroke and 75 were stroke mimics based on acute neurological symptoms and imaging findings. The CNN was trained on 20 non-stroke and 20 stroke patient CTA-SI volumes with manually delineated lesion targets. This was equal to 1400 axial images of which 20% included an ischemic lesion. Additional five non-stroke and five stroke volumes were used as validation data. Two test data sets were used, both consisting of 25 stroke and 25 stroke mimic cases. The volumes in both training and inference were resampled to isotropic 0.5×0.5×0.5 mm³ resolution. Cases used in CNN training were not part of the study population for this study. The CNN consisted of a two-channel input, 40 3D convolutional layers with 3×3×3 kernel size, 16 filters each and valid padding, followed by a fully connected layer with 50 neurons and two output neurons (lesion/background) and softmax activation, producing voxel-by-voxel lesion presence confidences. Skip connections passing single layers were used to encourage gradient propagation. The network was fed with 147×147×147-voxel sub-volumes with equal number of stroke lesion positive and negative sub-volumes in each batch. The second input channel was the corresponding (left-right-mirrored) sub-volume from the contra-lateral hemisphere. The model was trained using batch-size eight and Adam optimizer for 30 epochs after which the validation loss, calculated on the separate set of 10 CTA-SI volumes, stopped improving. The network was implemented using Keras library version 2.2.4 and Tensorflow version 1.12.0.

Study design

Lesion volumes from CNN outputs and manual segmentations of final infarcts were calculated from all lesions in the affected cerebral hemisphere. Computed tomography perfusion-RAPID ischemic core estimations were reported as calculated by the software.

Only lesions in the affected cerebral hemisphere detected by the CNN were selected for the analysis with a volume threshold of >0.1 mL and a probability threshold of 0.5 for lesion inclusion. False positive lesions in the contralateral hemisphere or cerebellum were discarded from the analysis. This approach was chosen because in LVO, the site of arterial occlusion, and thus the affected hemisphere is readily identifiable from CTA.

Convolutional neural network performance was compared against a commercial software (RAPID, iSchemaView) in determining the infarct core volume derived from CTP as this is a validated and widely used method for treatment selection. The effect of two clinically relevant time windows (0–6 h and 6–24 h from symptom onset to start of CT protocol) on CNN and CTP-RAPID output accuracy in final infarct volume prediction was also tested. Patients whose time of symptom onset could not be accurately inferred from patient history (n = 32), were assigned to the 6–24 h time window. These were patients with either a wake-up stroke or patients who were found with stroke symptoms and who were last known to be well ≤24 h from presentation to the emergency department.

ASPECTS anatomical regions were visually evaluated to determine CNN performance regarding anatomical accuracy against expert segmentation, that is, did the CNN predicted lesions’ locations match the final infarct locations within the ASPECTS regions in the middle cerebral artery territory. Individual regions were labeled “positive” or “negative” for ischemic changes by a radiologist (LH), as determined by the CNN from acute phase CTA and by manual segmentations from follow-up CT. Manual segmentations were considered as ground truths. Accuracy, sensitivity, specificity, and Sørensen-Dice similarity coefficient were calculated from the regions’ true or false labeling.

Convolutional neural network was compared to CTP-RAPID in determining patient eligibility for EVT according to criteria from the DAWN trial (DWI or CTP Assessment with Clinical Mismatch in the Triage of Wake Up and Late Presenting Strokes Undergoing Neurointervention) in patients that presented in the 6–24 h time window. The cutoff points were ischemic core volumes ≤20 mL, ≤30 mL, or ≤50 mL depending on patient age and National Institutes of Health Stroke Scale score. Volume outputs from the CNN and CTP-RAPID were compared according to Figure 1. The number of true positives, true negatives, false positives, and false negatives was then calculated and sensitivity, specificity, negative, and positive predictive value for the CNN prediction were derived.

Figure 1.

Accuracy of the convolutional neural network (CNN) in triaging patients for endovascular thrombectomy (EVT) was assessed by defining CNN results as true positives, true negatives, false negatives, or false positives using criteria from the DAWN-study. Sensitivity, specificity, negative, and positive predictive value for the CNN prediction were then derived.

Statistical analysis

A linear model was fitted between the CNN derived volume outputs, manually segmented final infarct volumes, and CTP-RAPID ischemic core volumes (defined by cerebral blood flow (CBF) < 30%). Pearson correlation coefficients (r) were calculated to evaluate the correlation of CNN and CTP-RAPID derived volumes against final infarct volumes and CNN derived volumes to CTP-RAPID core volumes. Bland-Altman plots of agreement between infarct volume estimates and final infarct volumes and between CNN and CTP-RAPID derived estimates were also calculated. Calculations were performed using MATLAB version 2018b (MathWorks, Natick, MA, USA). Intraclass correlation coefficient (ICC) estimates and their 95% confidence intervals were calculated using SPSS Statistics for Windows, version 27.0 (IBM Corp., Armonk, NY) based on a single rater/measurement, absolute agreement, two-way mixed-effects model.

Results

A total of 117 patients met the inclusion criteria. Patients with hemorrhagic transformation of infarct resulting in parenchymal hematoma with mass effect were excluded, leaving 89 patients for the analysis. Mean infarct volume estimates for CNN and CTP-RAPID outputs as well as final infarct volumes are presented in Table 2. No discernible infarct was found in 10 patients on follow-up imaging.

Table 2.

Infarct lesion volumes provided by the CNN, CTP-RAPID, and measurements from follow-up imaging in mL, mean (SD, range).

	All cases (n = 89)	0–6 h time window (n = 51)	6–24 h time window (n = 38)
CNN output (mL)	54 (45, 0–183)	58 (49, 0–183)	48 (39, 0–177)
CTP-RAPID infarct core (mL)	28 (36, 0–207)	31 (33, 0–106)	24 (39, 0–207)
Final infarct volume (mL)	36 (58, 0–358)	32 (50, 0–209)	41 (69, 0–358)

CNN: convolutional neural network, CTP-RAPID: Computed tomography perfusion RAPID.

Among all cases, irrespective of time from symptom onset, CTP-RAPID showed a tendency for more accurate estimates of final infarct volumes (r = 0.69, slope 1.1, p < 0.001), while the CNN had a tendency for overestimation (r = 0.50, and per linear regression, final infarct volume = 0.7 × CNN volume +1.0 mL, p < 0.001) as shown (Figures 2 and 3). The CNN underestimated infarct volumes compared to final infarct volumes in 28% of cases with a mean volume difference of 38.9 mL (SD 44.7 mL). Compared to CTP-RAPID, the CNN underestimated infarct volumes in 11% of cases with a mean volume difference of 13.9 mL (SD 12.5 mL).

Figure 2.

Lesion volume (mL) correlation between convolutional neural network (CNN) output and manual segmentation from follow-up imaging. (a) All cases (n = 89). (b) Patients imaged <6 h from symptom onset (n = 51). (c) Patients imaged 6–24 h from symptom onset (n = 38). (d) Bland-Altman plot of agreement between lesion volume estimates of the CNN based on acute phase CT angiography (CTA) and final infarct volumes in the 6–24 h time window.

Figure 3.

Lesion volume (mL) correlation between CT perfusion RAPID (CTP-RAPID) ischemic core (CBF <30%) and manual segmentation from follow-up imaging. (a) All cases (n = 89). (b) Patients imaged <6 h from symptom onset (n = 51). (c) Patients imaged 6–24 h from symptom onset (n = 38). (d) Bland-Altman plot of agreement between lesion volume estimates of CTP-RAPID and final infarct volumes in the 6–24 h time window.

Correlations with final infarct volumes in the early 0–6 h time window were r = 0.43 (p = 0.002) for the CNN and r = 0.58 (p < 0.001) for CTP-RAPID. Computed tomography perfusion-RAPID showed a trend for more accurate volume estimates (slope 0.9 vs. 0.4 for the CNN) as shown (Figures 2 and 3).

In the late 6–24 h time window, a better correlation with final infarct volumes was found for both the CNN (r = 0.67, slope 1.2, p < 0.001) and CTP-RAPID (r = 0.82, slope 1.4, p < 0.001) (Figures 2 and 3). Both methods showed a trend for underestimating infarct volumes. The mean volume difference between CNN output and final infarct volume was 7 mL (95% limits of agreement −145–81 mL). The mean volume difference between CTP-RAPID and final infarct volume was −18 mL (95% limits of agreement −152–34 mL).

Intraclass correlation coefficients for the comparisons above are presented in Table 3 suggesting moderate reliability between CNN output and final infarct volume in the late time window.

Table 3.

Reliability of the convolutional neural network (CNN) and CT perfusion RAPID (CTP-RAPID) in predicting final infarct volume, intraclass correlation coefficients and their 95% confidence intervals.

	Intraclass correlation	95% confidence interval	p-value
All cases
CNN Output versus final infarct volume	0.46	0.28–0.61	<0.001
CTP-RAPID versus final infarct volume	0.61	0.47–0.73	<0.001
0–6 h time window (n = 51)
CNN Output versus final infarct volume	0.38	0.11–0.59	<0.001
CTP-RAPID versus final infarct volume	0.54	0.32–0.71	<0.001
6–24 h time window (n = 38)
CNN Output versus final infarct volume	0.58	0.32–0.76	<0.001
CTP-RAPID versus final infarct volume	0.67	0.44–0.82	<0.001

CNN: convolutional neural network, CTP-RAPID: Computed tomography perfusion RAPID.

A correlation of r = 0.73 (slope 0.6, p < 0.001) was found between CNN and CTP-RAPID outputs when all patients, irrespective of time of symptom onset, were included in the analysis, with the CNN having a tendency to overestimate infarct sizes compared to CTP-RAPID (Figure 4). In subgroup analyses using the early and late time windows, correlations between CNN and CTP-RAPID were r = 0.72 (slope 0.5, p < 0.001) and r = 0.77 (slope 0.8, p < 0.001), respectively. In the late time window, the mean volume difference between CNN and CTP-RAPID was 25 mL (95% limits of agreement −27–82 mL).

Figure 4.

Lesion volume (mL) correlation between the convolutional neural network (CNN) output and CT perfusion RAPID (CTP-RAPID) ischemic core (CBF <30%). (a) All cases (n = 89). (b) Patients imaged <6 h from symptom onset (n = 51). (c) Patients imaged 6–24 h from symptom onset (n = 38). (d) Bland-Altman plot of agreement between lesion volume estimates of the CNN and CTP-RAPID in the 6–24 h time window.

Convolutional neural network accuracy in triaging patients for EVT in the late time window was compared to CTP-RAPID and assessed according to criteria from the DAWN trial as described in the statistical analysis section. Compared to CTP-RAPID, the CNN had a sensitivity of 0.38 with a specificity of 0.89, negative predictive value of 0.31, and positive predictive value of 0.92.

Among all patients, 11 had a chronic cerebral infarct on the same side as the LVO. In three of these cases, the CNN marked parts of the chronic infarct as an acute ischemic lesion. However, this changed the volume prediction only by 0.2–3 mL. A total of 890 ASPECTS regions were evaluated to determine the anatomical accuracy of the CNN. The accuracy, sensitivity, and specificity were 0.62, 0.70, and 0.57, respectively. The Sørensen-Dice similarity coefficient was 0.60.

Discussion

Better correlation between our CTA-based CNN outputs and CTP-RAPID core estimates with final infarct volumes were found in the late (6–24 h) versus early (0–6 h) time window. This finding supports the notion of Goyal et al. that extensive use of perfusion imaging in the early time window might not be desirable, and that instead, clinical-imaging mismatch (using NCCT) should be considered to be used for penumbra estimation while using CTA for LVO detection. In the same vein, Lopez-Rivera et al. found an increased likelihood for undergoing EVT in centers with lower CTP utilization, which was not associated with worse clinical outcomes or increased hemorrhage, suggesting under-treatment bias with routine CTP. Also, Boned and Martins have described the “ghost infarct core,” which refers to the tendency of CTP to overestimate infarct core size in an early time window of <3 h from symptom onset.[14,15]

Correlation with realized infarcts was moderate with the CNN and good with CTP-RAPID in the late time window. Previously, we found a good correlation between CNN outputs and final infarct volumes in patients with anterior circulation AIS that were not treated with EVT. In that study, 55% of patients were treated with supportive care and 45% received thrombolytic therapy, which may explain the differences in performance, as a large proportion of patients can be assumed to have suffered from infarct growth and several studies have shown that CTA is more CBF than CBV weighted including some penumbra.[16-18] On the other hand, Bal et al. found CTA-SI ASPECTS better than NCCT at predicting final infarct extent, especially in a very early 0–90 min time window. In another study, Sallustio et al. found CTA-SI ASPECTS to be a better predictor of outcome than NCCT in patients with stroke treated with EVT. In both of these studies, a fast image acquisition protocol was used for CTA. The conflicting results from these five studies could, at least in part, be explained with different ways of measuring infarct extent as three of the studies used ASPECTS and two used manually segmented lesion volumes to compare performance. These studies also used variable outcome measures for determining CTA performance and inclusion criteria for time from symptom onset to presentation also varied between studies from <3 h to <9 h. As such, findings from these studies may not reflect CTA performance in later time windows.

The observation that in the late time window, CTP-RAPID tended to underestimate core volumes, is in contradiction with numerous previous studies, which have shown a trend for core volume overestimation with CTP.[21-24] These studies, however, used delays of <9 h from symptom onset as inclusion criteria, so these findings might not reflect CTP performance with delays of >9 h. Moreover, large studies have shown the benefit of EVT up to 24 h after symptom onset using CTP-RAPID to guide patient selection.[1,2,25] However, CTP has also received critique in recent history, and it has been questioned, whether it should even be used outside of properly powered clinical trials. Possible problems in using CTP for treatment selection in individual patients include: optimal thresholds varying between vendors and postprocessing platforms, with time after stroke, quality of collateral flow, ischemic preconditioning, and duration of perfusion scans.[12,22,26-29] CTP is also more susceptible to motion artifacts than CTA.

Our CNN method had a positive predictive value of 0.92, that is, if CNN predicted core volume was below the cutoff point, CNN had a probability of 0.92 to correctly classify the patient as eligible for thrombectomy. However, the negative predictive value was only 0.31, which means that a number of patients that could benefit from EVT, would be left outside of treatment if only the CNN method was used.

Our CNN was trained with lesions that were manually segmented from CTA-SI as the ground truth. More accurate final infarct volume prediction may be possible by training the CNN with a different ground truth, such as manually segmented final infarcts from follow-up imaging. This introduces its own challenges, though. For example, there is no one optimal delay for follow-up imaging, as more than 30% relative growth in infarct volume can be witnessed in a significant portion of patients after 24 h and a 3–5 day delay may overestimate infarct size also due to ischemia related edema. Follow-up ischemic lesion volume at 24 h has nevertheless been found to be a valuable secondary outcome measure. The CNN underestimated final infarct volume in 28% of cases. In all of these cases, an underestimation was observed with CTP-RAPID also. This underestimation may be related to infarct progression and edema.

Sheth et al. used CTP-RAPID ischemic core estimates as ground truth for their DeepSymNet algorithm with good results but did not present comparisons to follow-up imaging. The results from their study and our previous study, however, suggest that it may be possible to get reasonably accurate infarct core estimations for triaging purposes using a CTA-based deep learning method. Hilbert et al. have also used CTA-based deep learning models in predicting functional outcome and reperfusion results in AIS. Their approach was quite different in that no lesion segmentations or volumetric data was used for neural network training, but instead, functional outcome and reperfusion measures were used as outcomes and visualization models were used afterward to assess which features or parts of the images the models used for decision making.

In this study, we selected consecutive patients who received EVT to simulate real life performance. This resulted in some limitations to available data, as the exact time from symptom onset to recanalization was unknown for almost half of all patients, albeit they presented ≤24 h from the time they were last known to be well. This prohibits us from analyzing whether correlation between CNN outputs and final infarct volumes would have been better depending on the delay from symptom onset to recanalization. We used follow-up CT to determine final infarct extent, as a 24 h follow-up CT is the standard protocol in our institution, although it is not as sensitive as DWI. Other limitations of this study include different vendors for CTA imaging and variation in follow-up time and imaging modalities.

In conclusion, a CTA-based CNN is able to detect anterior circulation ischemic strokes with moderate correlation to final infarct volumes in the late time window (6–24 h) in patients successfully treated with EVT.