Multicentre multiobserver study of diffusion-weighted and fluid-attenuated inversion recovery MRI for the diagnosis of sporadic Creutzfeldt-Jakob disease: a reliability and agreement study.

Objectives To assess the utility of the display standardisation of diffusion-weighted MRI (DWI) and to compare the effectiveness of DWI and fluid-attenuated inversion recovery (FLAIR) MRI for the diagnosis of sporadic Creutzfeldt-Jakob disease (sCJD). Design A reliability and agreement study. Setting Thirteen MRI observers comprising eight neurologists and five radiologists at two universities in Japan. Participants Data of 1.5-Tesla DWI and FLAIR were obtained from 29 patients with sCJD and 13 controls. Outcome measures Standardisation of DWI display was performed utilising b0 imaging. The observers participated in standardised DWI, variable DWI (the display adjustment was observer dependent) and FLAIR sessions. The observers independently assessed each MRI for CJD-related lesions, that is, hyperintensity in the cerebral cortex or striatum, using a continuous rating scale. Performance was evaluated by the area under the receiver operating characteristics curve (AUC). Results The mean AUC values were 0.84 (95% CI 0.81 to 0.87) for standardised DWI, 0.85 (95% CI 0.82 to 0.88) for variable DWI and 0.68 (95% CI 0.63 to 0.72) for FLAIR, demonstrating the superiority of DWI (p<0.05). There was a trend for higher intraclass correlations of standardised DWI (0.74, 95% CI 0.66 to 0.83) and variable DWI (0.72, 95% CI 0.62 to 0.81) than that of FLAIR (0.63, 95% CI 0.53 to 0.74), although the differences were not statistically significant. Conclusions Standardised DWI is as reliable as variable DWI, and the two DWI displays are superior to FLAIR for the diagnosis of sCJD. The authors propose that hyperintensity in the cerebral cortex or striatum on 1.5-Tesla DWI but not FLAIR can be a reliable diagnostic marker for sCJD.

Introduction

Reliable detection of Creutzfeldt–Jakob disease (CJD) is imperative for infection control and treatment. MRI is useful for the early diagnosis of CJD,1 2 whereas the utility of EEG and conventional cerebrospinal fluid (CSF) tests have been limited.3 Diffusion-weighted imaging (DWI) and fluid-attenuated inversion recovery (FLAIR) are key techniques for the diagnosis of sporadic CJD (sCJD) with high sensitivity and specificity when assessed by expert neurologists or neuroradiologists.1 4 5 However, the utility of MRI for general neurologists or radiologists who are not familiar with diagnosing CJD remains elusive. Moreover, standardisation of MRI methodologies, which would be essential for the reproducible assessment of MRI findings of CJD cases, has not been achieved in previous studies.1 4 5 DWI display conditions may particularly vary among institutions or operators,6 which can give rise to inaccurate assessment of CJD-related lesions, specifically subtle abnormalities in the cerebral cortex. Meanwhile, although DWI seems superior to FLAIR for the detection of CJD lesions,5 direct comparison of the two sequences is yet to be performed.

To address these issues, we investigated the utility of a newly proposed standardisation method of DWI display6 7 and compared the effectiveness of DWI and FLAIR for the diagnosis of sCJD, particularly for differentiating between abnormal and normal signals by neurologists and radiologists who are not necessarily CJD experts. We conducted a multicentre, multiobserver case–control study and evaluated observer performance with receiver operating characteristics (ROC) analysis.

Methods

Subjects

Patients diagnosed as having sCJD by the CJD Surveillance Committee of Japan8 from October 2005 to September 2010 were eligible to participate in this study. The accuracy of the diagnosis was defined as follows: definite, that is, pathologically verified cases; probable, that is, cases with neuropsychiatric manifestations compatible with sCJD and periodic sharp wave complexes on EEG without pathological examinations and possible, that is, cases with the same findings as probable sCJD but no periodic sharp wave complexes on EEG.8 9 WHO criteria10 were not applied because the assay of CSF 14-3-3 protein, which is required by WHO criteria, was standardised only since April 2009 in Japan.11 The prion protein gene (PRNP) was analysed in the open reading frame after extracting DNA from patients' blood.12 13 For neuropathological examinations, brain sections were stained with routine techniques, and immunohistochemistry was performed using the mouse monoclonal antibody 3F4 (Senetek, MD Heights, Missouri, USA).12 For PrPSc typing, frozen brain tissues were homogenised and analysed by western blot for proteinase K-resistant PrP using the 3F4 antibody.14 Assays for CSF γ-isoform of 14-3-3 protein,11 total τ protein (cut-off value, 1300 pg/ml)15 and real-time quaking-induced conversion (RT-QUIC)16 were performed in patients whose CSF samples were available. The following patients were eligible as disease controls: patients who were suspected to have prion disease by primary physicians but were denied to have prion disease by the Committee or those who were diagnosed as having other neurological disorders at Tokushima University Hospital and whose brain MRI showed no abnormal intensity in the cerebral cortex or striatum. We requested physicians who had referred patients to the Committee to provide initial MRI data of eligible patients.

This study was approved by the Medical Ethics Committee of Kanazawa University and the Ethics Committees of the Tokushima University Hospital and Tokyo Medical and Dental University. Written informed consent was obtained from all patients or their families.

Magnetic resonance imaging

DWI, b0 and FLAIR images were converted to the Digital Imaging and Communication in Medicine format. When the Digital Imaging and Communication in Medicine data contained patient information, it was excluded by one of the investigators (MH) before the observer performance study. All MRI studies were performed on 1.5-Tesla scanners at each hospital. Quadrature detection head coils or multichannel head coils were used. DWI was performed using the single-shot spin-echo echo planar imaging technique with the following parameters: repetition time, 4000–8000 ms; echo time, 70–100 ms; b value, 1000 s/mm2; slice thickness, 5 mm; matrix size, 128×80 to 128×128; field of view, 220–230 mm and 16–20 contiguous axial sections parallel to a line through the anterior and posterior commissures were obtained from each patient. The scanning parameters of FLAIR were as follows: repetition time, 8000–10 000 ms; inversion time, 2000–2500 ms; effective echo time, 105–120 ms; matrix size, 256×192 to 320×224; field of view, 210–220 mm; slice thickness, 5–6 mm with 1–1.5 mm interslice gaps and 19−20 slices per patient.

Display methods

Two display methods were used for DWI: standardised and variable. In the standardised display, the window width and level settings were constant for all evaluations and could not be changed. Details of the standardised display have been reported elsewhere.6 In brief, the window width and level were as follows: window width = SIb0 and window level = SIb0/2, where SIb0 represents the signal intensity in the normal-appearing subcortical region on b0 imaging.6 One radiologist (MH) manually measured SIb0 within a circular region of interest. The calculated window width and level were applied to all images. In the variable display mode, regarded as the most reliable for assessment of DWI, each observer was able to change the window width and level settings on the monitor according to preference. FLAIR was assessed with the variable display method because no standardised methods are currently available for FLAIR display.

Observer performance study

Eight neurologists (6–27 years of experience; mean, 12 years; board certified, six) and five radiologists (5–25 years of experience; mean, 12.8 years; board certified, four; neuroradiologist, one) participated in the observer performance study at The University of Tokushima Graduate School (persons, six; neurologists, three) and Tokyo Medical and Dental University (persons, seven; neurologists, five including NS and YS). Before the test, the observers were informed that the purpose of the study was to evaluate their performance in detecting MRI lesions compatible with CJD, that is, hyperintensity in the cerebral cortex or striatum, regardless of signal changes in other regions including the thalamus. Three sessions were conducted: standardised DWI, variable DWI and FLAIR. To reduce the effect of learning, the interval between reading sessions was 1 week or longer. The order of the three sessions was randomised among the observers. Using computer randomisation, images of patients with and without sCJD were intermixed. All cases were presented in the same randomised order to the observers in each session. Each observer independently viewed all slices of each MRI study on the same type of monitors (Let'snote, Panasonic, Osaka, Japan) using INTAGE Realia Professional (Cubernet, Tokyo, Japan) run on Windows XP (Microsoft). The observers were allowed to adjust the window width and level only in variable DWI and FLAIR sessions but not in the standardised DWI session. Observers were blinded to any clinical information including age, sex and diagnosis.

Each observer used a continuous rating scale of a line-marking method to rate his or her confidence level on the paper format independently. At the left end of the line, a confidence level that lesions compatible with CJD were definitely absent was indicated, whereas at the right end, a confidence level that lesions were definitely present was indicated. Intermediate levels of confidence were indicated by the different positions on the line between the two ends. The distance between the left end and the marked point was converted to a confidence level that could range from 0 to 100, as described elsewhere.17

Statistical analyses

Observer performance was evaluated using ROC analysis with SPSS V.19 (IBM). The ROC curves for each observer indicated the ratio of the true-positive fraction to the false-positive fraction at each confidence level. The area under the ROC curve (AUC) was used to compare observer performance for accurately detecting CJD lesions. Intraclass correlations were calculated in the neurologist group, the radiologist group and for all observers by two-way random consistency measures using the SPSS software. In all the analyses, p values of <0.05 were considered statistically significant.

Results

Patients and controls

MRI data from 85 patients were provided. Of these, 42 subjects from 15 hospitals (including authors' institutions) were eligible for this study after excluding cases that were diagnosed as having non-sporadic CJD or lacked the required MRI sequences. This study cohort included 29 patients with sCJD (men, 11; mean age, 71 years; duration before MRI, 4.4±6.1 months), three patients who were suspected of CJD but eventually diagnosed as negative by the Committee and 10 patients diagnosed as having other neurological disorders. Of the 29 sCJD patients, four had definite, 24 had probable and one had possible CJD. Twenty-six cases underwent PRNP analysis, 24 were homozygous for methionine at codon 129 and two were heterozygous with methionine and valine at codon 129. Of the four definite cases, PrPSc was type 1 in two cases, type 1+2 in one case and type 2 in one case. Eleven of 15 CSF samples from probable sCJD cases were positive for PrPSc by RT-QUIC (table 1).

Table 1

Clinical profiles of patients with sporadic Creutzfeldt–Jakob disease

No	Age/sex	Diagnosis	Codon129/PrPSc	14-3-3/total τ	RT-QUIC	Pre-MRI duration (months)
1	69/M	Definite	MM/1	+/+	+	−2*
2	77/F	Definite	MM/1	+/+	+	19
3	75/F	Definite	ND/1+2	+/+	+	3
4	65/M	Definite	MM/2	−/−	−	12
5	69/M	Probable	MM	ND	−	0.5
6	72/F	Probable	MM	ND	ND	0.5
7	77/F	Probable	MM	−/−	+	0.5
8	72/M	Probable	MM	ND	ND	1
9	63/M	Probable	MM	+/+	+	1.5
10	88/F	Probable	MM	ND	ND	1.5
11	75/M	Probable	MV	ND	ND	1.5
12	56/M	Probable	MM	+/+	+	2
13	67/M	Probable	MM	+/+	−	2
14	70/M	Probable	MM	+/+	+	2
15	70/F	Probable	MM	+/+	+	2
16	74/F	Probable	MM	+/+	−	2
17	84/F	Probable	MM	+/−	+	2
18	85/F	Probable	MM	−/+	+	2
19	49/F	Probable	ND	+/+	+	2
20	74/F	Probable	MV	+/+	+	2.5
21	54/F	Probable	ND	ND	ND	2.5
22	61/M	Probable	MM	ND	ND	3
23	72/F	Probable	MM	+/−	−	3
24	81/F	Probable	MM	−/−	+	3
25	70/M	Probable	MM	+/+	+	6
26	83/F	Probable	MM	+/+	ND	9
27	67/F	Probable	MM	+/+	−	15
28	84/F	Probable	MM	+/+	−	26
29	57/F	Possible	MM	ND	ND	4

MRI was obtained 2 months before the symptom onset.2

MM, homozygous for methionine; MV, heterozygous with methionine and valine; ND, not done; RT-QUIC, real-time quaking-induced conversion.

Diagnoses for three prion-denied patients were immune-mediated encephalopathy, juvenile Alzheimer's disease and frontotemporal dementia. Other neurological controls were diagnosed with Alzheimer's disease, Parkinson's disease, spinocerebellar degeneration, vascular dementia, old cerebral infarction, benign paroxysmal positional vertigo, dizziness, temporal arteritis, cervical spondylosis and diabetic neuropathy.

Diagnostic performance

We investigated the diagnostic performance of standardised DWI, variable DWI and FLAIR images assessed by eight neurologists and five radiologists using ROC analysis. Mean AUC values obtained from the three sessions were compared within the neurologist group, the radiologist group and all observers (figure 1, table 2). The AUC values for standardised and variable DWI were not different within each professional group or for the total observer group. On the other hand, AUC values for FLAIR were significantly lower than DWI displayed by either method (p<0.05). Representative MRI scans are shown in figure 2.

Figure 1

Receiver operating characteristic curves for each display in diagnosis of sporadic Creutzfeldt–Jakob disease. (A) Neurologists, (B) radiologists and (C) all observers. The true rate (sensitivity) is plotted as a function of the false-positive rate (1 − specificity). DWI, diffusion-weighted imaging; FLAIR, fluid-attenuated inversion recovery; sDWI, standardised DWI; vDWI, variable DWI.

Table 2

Areas under the receiver operating characteristic curves

	Neurologists	Radiologists	All observers
sDWI	0.86 (0.82 to 0.90)	0.82 (0.77 to 0.88)	0.84 (0.81 to 0.87)
vDWI	0.86 (0.82 to 0.90)	0.83 (0.77 to 0.89)	0.85 (0.82 to 0.88)
FLAIR	0.69 (0.63 to 0.75)	0.66 (0.58 to 0.73)	0.68 (0.63 to 0.72)

Means (95% CIs) are indicated.

DWI, diffusion-weighted imaging; FLAIR, fluid-attenuated inversion recovery; sDWI, standardised DWI; vDWI, variable DWI.

Figure 2

Representative MRI of a sporadic Creutzfeldt–Jakob disease patient (case 17). Abnormal hyperintensity in the cerebral cortex is evident on standardised diffusion-weighted imaging (A, arrow) but obscure on fluid-attenuated inversion recovery (B, arrow).

Rating agreement

To measure the extent to which the observers agreed when rating the MRI findings, intraclass correlations were calculated. The intraclass correlations of the standardised DWI (0.74, 95% CI 0.66 to 0.83) and variable DWI (0.72. 95% CI 0.62 to 0.81) tend to be higher than that of FLAIR (0.63, 95% CI 0.53 to 0.74), specifically in the neurologist group, although the differences were not significant (figure 3).

Figure 3

Intraclass correlations for each display. Error bars represent upper limits of 95% CIs. DWI, diffusion-weighted imaging; FLAIR, fluid-attenuated inversion recovery; sDWI, standardised DWI; vDWI, variable DWI.

Discussion

We demonstrated that standardised DWI was as useful as variable DWI and that both DWI displays are superior to FLAIR for the diagnosis of sCJD when assessed by multiobservers with various specialty backgrounds.

Our standardisation method of DWI display was originally proposed as an easy-to-use way to decide the window width and level for DWI even in emergency settings.6 Indeed, this method was demonstrated as useful for detecting acute ischaemic lesions on DWI.7 Results of the present study show that the standardisation method is also reliable for diagnosis of sCJD, in which DWI is one of the key sequences. We suggest some advantages of standardised DWI over variable DWI, although there was no statistical difference between the two methods. First, standardised DWI can be helpful for physicians who can refer only to hardcopies but not softcopies. Second, even for doctors who can readily refer to softcopies and thus variable DWI, the standardisation method can simplify assessment procedure without any disadvantages. Third, the standardisation can facilitate direct comparison of DWI findings from different CJD patients.

DWI and FLAIR have been reported as useful markers for the diagnosis of CJD. Of these, DWI has been assumed to be the most sensitive, although without direct evidence.1 5 18 Hyperintensity in the cerebral cortices, the striatum or both indicates the diagnosis of CJD. The striatum hyperintensity is anterior dominant at early stages of the disease.19 MRI lesion profiles reportedly differ among molecular subtypes of sCJD,20 21 which was not reproduced in a recent study.5 Zerr et al4 proposed that high-signal abnormalities in caudate nucleus and putamen or at least two cortical regions (temporal, parietal or occipital lobes) either in DWI or FLAIR together with typical clinical signs can be diagnostic for probable sCJD. Based partly upon their report, ‘high signal in caudate/putamen on MRI brain scan’ has been used as one of the laboratory findings in the diagnostic criteria for probable sCJD in the European CJD Surveillance System (EUROCJD) since January 2010.22 However, their criteria did not distinguish DWI and FLAIR, thereby maintaining ambiguity about the diagnostic values of MRI in situations where DWI is not available. Our data indicate that FLAIR without DWI is unreliable for the diagnosis of sCJD. On the other hand, high signals in the cerebral cortex have not been regarded as diagnostic in the EUROCJD criteria, probably because cortical abnormalities are less reliable on conventional MRI. Our results suggest that, using standardised or variable DWI but not FLAIR, cortical signals can also be used as a diagnostic marker.

Meanwhile, Young et al23 reported that the sensitivity and the specificity of DWI and FLAIR for the diagnosis of CJD are 91% and 95%, respectively. More recently, Vitali et al5 reported that hyperintensity greater on DWI than FLAIR is diagnostic for sCJD, whereas hyperintensity greater on FLAIR than DWI is characteristic for non-prion rapidly progressive dementia. Furthermore, reduction of apparent diffusion coefficient in subcortical (striatum) hyperintensity regions on DWI is supportive for sCJD.5 24 25 These findings can be greatly helpful for differentiating sCJD from other rapidly progressive dementia. However, assessment of FLAIR lesions tends to vary among physicians, particularly among neurologists, as shown by the present study, and standardised methods for FLAIR or apparent diffusion coefficient map have not been established until date. Therefore, clinical criteria which require DWI but not necessarily FLAIR or apparent diffusion coefficient will be more readily applicable.

As many as 13 neurologists and radiologists from different institutions participated in the observer performance study, although the sample size of patients was relatively small. Notably, the observers had various specialty backgrounds such as stroke neurologists, neurophysiologists, experts in dementia or prion disease and general and neuroradiologists. This variety simulates practical situations in which the diagnosis of suspected CJD cases may be made by physicians who do not necessarily specialise in prion disease.

This study has some limitations. First, we did not evaluate patterns of cortical involvement suggestive of sCJD4 5 because we had to address whether DWI or FLAIR is suitable for detecting cortical lesions in the first place. Second, we did not assess the difference among sCJD subtypes21 because majority of our cases had a typical phenotype and were homozygous for methionine; thus, they were compatible with MM1 sCJD. Until date, MM2 thalamic-type sCJD remains a diagnostic challenge in MRI-based assessment; thalamic hypoperfusion or hypometabolism on SPECT or PET can be useful.26 Third, majority of the control patients were not those who were suspected to have CJD. However, the principle aim of the present study was to establish a display method, which reliably distinguishes potentially CJD-associated signals from normal signals. Thus, our results provide a practical foundation for utilising DWI as a general diagnostic marker of sCJD when combined with previous findings.1 4 5

Although neuropathological confirmation of the diagnosis of sCJD was obtained in few cases, we performed RT-QUIC, a newly established CSF PrPSc amplification assay which achieved >80% sensitivity and 100% specificity for CJD.16 Overall, 15 of 29 cases (51.7%) were pathologically proven or confirmed by RT-QUIC to have CJD. There were no significant differences in MRI findings between sCJD patients with and without positive results of CSF 14-3-3 protein, total τ protein or RT-QUIC. It will be important to further evaluate accurate diagnostic ability (sensitivity and specificity) of DWI in a prospective cohort of suspected CJD patients, that is, consecutive patients registered to the CJD surveillance who will also undergo CSF confirmation tests or neuropathological analyses.

In conclusion, we suggest that hyperintensity in the cerebral cortex or striatum assessed on the standardised or variable DWI scanned with 1.5-Tesla machines can be a reliable first-line on-site diagnostic marker for sCJD.