Inter-rater reliability of modified Alberta Stroke program early computerized tomography score in patients with brain infarction.

BACKGROUND: The Alberta Stroke Program Early Computerized Tomography Score (ASPECTS) was used to detect significant early ischemic changes on brain CT of acute stroke patients. We designed the modified ASPECTS and compared it to the above system based on the inter-rater reliability.
METHODS: A cross-sectional validation study was conducted based on the inter-rater reliability. The CT images were chosen from the stroke data bank of Ghaem hospital, Mashhad in 2010. The inclusion criteria were the presence of middle cerebral artery territory infarction and performance of CT within 6 hours after stroke onset. Axial CT scans were performed on a third-generation CT scanner (Siemens, ARTX, Germany). Section thickness above posterior fossa was 10 mm (130 kV, 150 mAs). Films were made at window level of 35 HU. The brain CTs were scored by four independent radiologists based on the ASPECTS and modified ASPECTS. The readers were blind to clinical information except symptom side. Cochrane Q and Kappa tests served for statistical analysis.
RESULTS: 24 CT scans were available and of sufficient quality. Difference in distribution of dichotomized ≤7 and >7 ASPECT scores between four raters was significant (Q=13.071, df=3, p=0.04). Distribution of dichotomized <6 and ≥6 scores based on modified ASPECT system between 4 raters was not significantly different (Q=6.349, df=3, p=0.096).
CONCLUSIONS: Modified ASPECT method is more reliable than ASPECTS in detecting major early ischemic changes in stroke patients candidated to tPA thrombolysis.

While the use of tissue plasminogen activator (tPA) in selected stroke patients is endorsed by international stroke guidelines, one concern with the use of this therapy is the risk of precipitating parenchymal hematoma with mass effect1. In middle cerebral artery (MCA) strokes, involvement of greater than one third of the territory by ischemic tissue increases the risk of intracerebral hemorrhage2. The relationship between the extend of ischemia and thrombolysis-related hemorrhage has received support from several recent studies.123 Since early signs of major MCA territory infarction are considered as an exclusion criteria in thrombolysis therapy, physicians must be able to reliably identify scans with this finding.3 However, estimation of ischemic tissue extent is hampered by modest interobserver agreement among experts, with reported k-values of 0.39-0.644. Additionally, it appears that clinicians have a tendency to rate very small and large infarcts well but have trouble with classifying moderately large infarcts.4 The Alberta Stroke Program Early CT Score (ASPECTS) system was used for estimation of large MCA territory infarcts which are ineligible to thrombolysis with tPA. The ASPECTS is semi-quantitative and localization weighted estimation of ischemic tissue volume within MCA territory. This method improves detection of early ischemic changes in the CT scan. We designed modified ASPECTS as an alternative to the above system to increase the interobserver agreement among experts. This article compares interobserver agreement among four radiologists between ASPECTS and modified ASPECTS methods in patients with hyperacute brain infarction

Methods

This cross-sectional study was conducted in ischemic stroke patients admitted to Ghaem hospital, Mashhad during 2010. The CT images were chosen from Ghaem stroke data bank. The inclusion criteria were the presence of MCA territory infarction and availability of electronic and hard copies of CT scans performed within 6 hours after stroke onset. Patients were excluded if infarction outside the MCA territory was present on CT image. Uncertain time of stroke onset and poor scan quality were considered as exclusion criteria. CT scans of stroke patients with lacunar syndrome were excluded. All consecutive brain CT images that met the above inclusion criteria and had no exclusion criteria were enrolled in this study. The human research ethics committee of Ghaem hospital approved the study. Axial CT scans were performed on a third-generation CT scanner (Siemens, ARTX, Germany) oriented along the supraorbital meatal line. Section thickness was 5 mm in the posterior fossa (130 kV, 150 mAs) and 10 mm superiorly (130 kV, 150 mAs). Films were made at window level of 35 HU. All patient details were removed from the films. Early signs of ischemia were defined as X-ray hypoattenuation, loss of the gray-white boundary (which is due to X-ray hypoattenuation of the gray matter) and focal brain swelling.5 The later was defined as any focal narrowing of the cerebrospinal-fluid space due to compression by adjacent brain structures such as effacement of the cortical sulci or ventricular compression5. Only new areas of ischemia were scored. The ASPECTS system is an alternative method for detection of more than one third involvement of MCA territory.67 In ASPECTS, the entire MCA territory is summarized by 10 regions of interest on two axial CT slices which involve basal ganglionic and supraganglionic structures67 (Figure 1). One point is deduced for partial or total involvement by ischemic tissue in any of the 10 designated regions.67 An ASPECT score of ≤7 is associated with an increased rate of symptomatic hemorrhage following intravenous tPA therapy.67 A normal CT scan has an ASPECTS value of 10 points. The modified ASPECTS system was designed by a stroke neurologist. The modified ASPECTS is calculated from two standard axial cuts, one at the level of the thalamus and basal ganglia, and one just rustral to the ganglionic structures.8 For modified ASPECTS, the territory of MCA is allotted 8 points. Modified ASPECTS is assessed by systematically scoring each of 7 regions (M1-M7) on the CT scan.8 The evaluated cortical regions M1 to M6 are the same as those of ASPECTS system (Figure 1). The M7 region includes subcortical structures (internal capsule, lentiform nucleus, external capsule, and insular ribbon) and insular cortex.8 The caudate head is not included in modified ASPECTS because it has dual blood supply from the MCA and anterior cerebral artery. One point is deduced for partial or total involvement by ischemic tissue in any of the M1 through M6 designated regions and two points is deduced in partial or total M7 region involvement.8 A score of 8 implies no evidence of new early signs of ischemia in the MCA territory. A progressively lower score indicates more extensive ischemic changes.8 A validation study was conducted for detection of interobserver agreement in estimating the volume of MCA stroke based on the ASPECTS and modified ASPECTS methods. Four experienced academic radiologists independently evaluated the CT images based on the ASPECTS and modified ASPECTS at separate sessions. The readers were given only brief training, which included a set of written guidelines based on relevant published articles67 and 5 separate CT films (2 with extensive early ischemic changes) shown for them as example.9 Rating for each method was performed 2 weeks apart9. Training on the use of each method was provided prior to each reading. The readers were blind to all clinical information except symptom side. All images were reviewed on film. The modified ASPECTS and ASPECTS templates was provided to the readers. Modified ASPECTS and ASPECTS scores were dichotomized at <6 or ≥6 and at ≤7 or >7, respectively.67 Data were entered into a SPSS16 software package. Cochrane Q test and Kappa test served for statistical analysis to compare the readers’ agreement for quantification of early CT ischemic changes based on the three mentioned methods.

Figure 1

The Alberta Stroke Program Early Computerized Tomography Score (ASPECTS) and modified ASPECTS templates.

C=caudate; L=lentiform; IC=internal capsule; I=insular ribbon; MCA= middle cerebral artery; M1=anterior MCA cortex; M2=MCA cortex lateral to insular ribbon; M3=posterior MCA cortex; M4, M5, and M6 are anterior, lateral and posterior MCA territories immediately superior to M1, M2 and M3, rostral to basal ganglia

Results

50 CTs were available while 16 scans excluded for poor quality or uncertain time of onset. 10 patients had brain infarction outside the MCA territory and excluded. 24 patients (15 males) of mean age of 61.8 years (SD: 8.5) with infarcts in the MCA territory were studied. 24 scans were examined by 4 raters yielding a total of 96 patient-scan examinations. These 24 scans which differently identified by the two methods belonged to the same 24 patients. Table 1 represents the inter-rater agreement of major MCA territory infarction based on the reading methods.

Table 1

Inter-rater agreement of major middle cerebral artery territory infarction based on the reading methods in 24 patients.

Four radiologists agreed on dichotomized ≤7 and >7 scores based on the ASPECTS method of 4 (16.6%) and 4 (16.6%) patients, respectively. 33.3% of the patients had whole raters agreement based on the ASPECTS method. Difference in distribution of dichotomized ≤7 and >7 ASPECT score among four raters was significant (Q=13.071, df=3, p=0.04). Four radiologists agreed on dichotomized <6 and ≥6 scores based on the modified ASPECTS method in 4 patients (16.6%) and 7 patients (29.2%), respectively. 45.8% of the patients had whole raters agreement based on the modified ASPECTS method. Difference in distribution of dichotomized <6 and ≥6 scores based on modified ASPECTS method among 4 raters was not significant (Q=6.349, df=3, p=0.096).

96 patient-scan examinations by 4 raters were analysed for number of patients considered as eligible for thrombolysis based on the two methods. 52.1% and 51.1% of 96 patient-scan examinations were ineligible to tPA therapy based on the ASPECTS and modified ASPECTS methods, respectively. Distribution of patients ineligible for thrombolysis was not significantly different comparing the ASPECTS and modified ASPECTS methods with each other (k=0.010, p=0885).

Discussion

The European co-operative Acute Stroke Study trial identified importance of early ischemic changes as a trend to increased mortality with rtPA treatment of patients with early ischemic changes visible in greater than one third of MCA territory.10 These stroke patients were treated up to 6 hours after symptom onset.10 Although intravenous thrombolysis with rTPA is approved within 3 hours post-event, intra-arterial rTPA therapy extends therapeutic time window up to 6 hours. Thus, interobserver reliability studies usually assess inter-rater agreement in brain CT obtained up to 6 hours after symptom onset.910 While MRI , especially diffusion and perfusion weighted images are highly sensitive for detection of early ischemic changes within 3 to 6 hours windows,4 CT is the routine imaging modality in candidates of rTPA therapy around the world11. Interobserver agreement study based on MRI is highly expensive and this was limitation of our research project. ASPECTS and modified ASPECTS were devised to provide a systematic approach of assessing and improving reliability of early ischemic changes in regions of the MCA territory. These systems provide a reliable semi-quantitative, localization weighted estimation of ischemic tissue volume within the MCA territory.412 ASPECTS is a predictor for both functional outcome and symptomatic hemorrhage in patients treated by rTPA thrombolysis.13 In an individual with ASPECTS value of 7 or less, the risk of symptomatic intracerebral hemorrhage with rTPA is 14 times that of patients with a score greater than 7.1314 The ASPECTS has shown higher inter-rater agreement than 1/3 MCA rule in some studies,1214 while similar reliability studies have demonstrated reverse results.915 Our study revealed superior reliability of modified ASPECTS than ASPECTS method. A significant difference in distribution of stroke patients ineligible to rTPA therapy was found between our four radiologists based on ASPECTS method, while this difference was not significant based on the modified ASPECTS system. 96 scan-patient examinations in our study group revealed insignificant difference in distribution of patients ineligible for rTPA thrombolysis comparing the ASPECTS and modified ASPECTS systems. These findings confirm that modified ASPECTS is an alternative to ASPECTS system which has higher reliability. This is not a surprising finding, since there are fewer number of measures, one will expect a better reliability. However, the most important result is not just inter-rater reliability, but whether the new method more accurately identifies patients who have adverse events following rtPA. Recent studies highlighted the importance of modification of ASPECTS, e.g., calculation of deep white matter ischemic lesions in ASPECTS.16 Another modification of ASPECT score has been developed on Diffusion/Perfusion MRI.17 The 10 designated regions on this modified ASPECT score is the same as initial ASPECTS system, however each region was rated as: 1= uninvolved, 2= possibly involved, 3= <50% involved, and 4= >50% involved.17 This modification of ASPECTS based on MRI was also validated with Diffusion/Perfusion MRI of 18 patients within 6 hours post event which did not contain more patients than our study.17 This type of ASPECTS modification was designed using Diffusion/Perfusion MRI which is not available in most hospitals and does not seem to simplify initial ASPECTS system. Our simplified modification of ASPECTS uses CT which is accessible in every hospital. Brain CT is the routine imaging in hyperacute phase of stroke for candidates of thrombolysis therapy and imaging exclusion criteria; e.g., intracerebral hemorrhage and extensive early ischemic changes are investigated routinely based on CT images.112 Additional training with ASPECTS is another solution to the problem of inter-rater reliability, however simplification of ASPECTS also helps to reach this purpose.

Conclusion

The modified ASPECTS was developed to offer the reliability and utility of a standard CT examination with a reproducible grading system to assess early ischemic changes in the MCA territory. This CT score has shown to be simple, reliable and identifies stroke patients unlikely to make an independent recovery despite thrombolytic treatment in our pilot study. However, the reliability of modified ASPECTS should be retested in further studies with higher number of samples and among other specialties including neurologists and stroke fellows.

Authors’ Contributions

Authors 1,10 designed the modified ASPECT and supervised the research

Authors 4,5,6,7 are the raters in the research

Author 3 performed the statistical analysis and interpretation of results

Authors 2, 8,9 hepled in collection of data and writting the paper