Diagnostic accuracy of semiautomatic lesion detection plus quantitative susceptibility mapping in the identification of new and enhancing multiple sclerosis lesions.

Purpose: To evaluate the diagnostic accuracy of a novel non-contrast brain MRI method based on semiautomatic lesion detection using T2w FLAIR subtraction image, the statistical detection of change (SDC) algorithm (T2w + SDC), and quantitative susceptibility mapping (QSM). This method identifies new lesions and discriminates between enhancing and nonenhancing lesions in multiple sclerosis (MS).
Methods: Thirty three MS patients who had MRIs at two different time points with at least one new Gd-enhancing lesion on the 2nd MRI were included in the study. For a reference standard, new lesions were identified by two neuroradiologists on T2w and post-Gd T1w images with the help of T2w + SDC. The diagnostic accuracy of the proposed method based on QSM and T2w + SDC lesion detection (T2w + SDC + QSM) for assessing lesion enhancement status was determined. Receiver operating characteristic (ROC) analysis was performed to compute the optimal lesion susceptibility cutoff value.
Results: A total of 165 new lesions (54 enhancing, 111 nonenhancing) were identified. The sensitivity and specificity of T2w + SDC + QSM in predicting lesion enhancement status were 90.7% and 85.6%, respectively. For lesions ≥50 mm3, ROC analysis showed an optimal QSM cutoff value of 13.5 ppb with a sensitivity of 88.4% and specificity of 88.6% (0.93, 95% CI, 0.87-0.99). For lesions ≥15 mm3, the optimal QSM cutoff was 15.4 ppb with a sensitivity of 77.9% and specificity of 94.0% (0.93, 95% CI, 0.89-0.97).
Conclusion: The proposed T2w + SDC + QSM method is highly accurate for identifying and predicting the enhancement status of new MS lesions without the use of Gd injection.

Introduction

Multiple sclerosis (MS) is a devastating autoimmune disease of the central nervous system characterized by inflammatory demyelination (Polman et al., 2011). MRI plays a central role in monitoring disease status, and is frequently performed on MS patients for routine surveillance and during suspected disease flares. Enhancing MS lesions identified on post-Gadolinium (Gd) T1w (T1wGd) images are typically used as an indicator of acute disease activity because they reveal areas of blood-brain barrier (BBB) disruption, a feature frequently present in acute lesions (Polman et al., 2011; Sommer et al., 2018; Yamamura et al., 2017). Therefore, the ability to identify these new enhancing lesions during follow-up imaging is of great importance in monitoring disease activity and informing therapeutic decision-making (Filippi et al., 2016; Wetter et al., 2016; Zivadinov et al., 2017).

Quantitative susceptibility mapping (QSM) is gaining interest as a novel quantitative tool to study MS, because pathological changes in myelin and iron cause measurable magnetic susceptibility changes in MS lesions (Chen et al., 2014; Eskreis-Winkler et al., 2015; Langkammer et al., 2013; Langkammer et al., 2012; Wisnieff et al., 2015). Histological studies show evidence of myelin breakdown and digestion by macrophages in actively demyelinating lesions, while iron tends to increase in chronic active lesions due to persistent microglial activation behind a sealed BBB (Gaitan et al., 2011; Kirschbaum et al., 2016; Mardiguian et al., 2017; Mehta et al., 2013; Wisnieff et al., 2015). Recent longitudinal studies (Chen et al., 2014; Zhang et al., 2016; Zhang et al., 2016) have found that new Gd-enhancing lesions tend to be isointense or slightly hyperintense on QSM. As these lesions transition from enhancing to nonenhancing, their susceptibility values increase over time, leading to a hyperintense appearance on QSM. Therefore, QSM lesion values can be used to predict lesion enhancement status, thus circumventing the need for qualitative visual assessment (Zhang et al., 2016). Given the recent concerns about Gd-deposition in the brain and calls to re-evaluate the need for Gd in specific clinical indications (Kanda et al., 2017; Ranga et al., 2017), these data suggest that QSM could play an important role in monitoring MS disease activity and inflammation without the use of Gd injections.

Despite the advantages offered by QSM, translating the emerging body of knowledge into routine clinical practice can be challenging, especially for new acute lesions which can be isointense relative to surrounding normal appearing white matter (NAWM) on QSM. Many automatic subtraction techniques have been developed to aid in the detection of new lesions between two time points using T2w (Battaglini et al., 2014; Cabezas et al., 2016; Ganiler et al., 2014; Horsfield et al., 2016; Kotari et al., 2018; van Heerden et al., 2015) or double inversion recovery (DIR) (Eichinger et al., 2017) images. Using this subtraction approach with co-registered QSM images might increase the accuracy and ease of characterizing new lesions in MS without Gd, but this has not been formally investigated.

In this study, we combined QSM with semiautomatic lesion detection using T2w subtraction to determine diagnostic accuracy when radiologists used this imaging strategy to perform two tasks: 1) detecting new MS lesions and 2) discriminating between new enhancing and new nonenhancing MS lesions.

Materials and methods

Patient population

This retrospective image analysis study was approved by the local institutional review board, which waived informed consent. All cases met the following inclusion criteria: 1) diagnosed with MS according to the 2010 McDonald Criteria with revision (Polman et al., 2011); 2) had at least 2 clinical brain MRIs performed within a 3-year interval, 3) had at least 1 new Gd-enhancing lesion on the 2nd MRI as described in the radiology report.

A total of 33 MS patients (14 male, 19 female) were included in this study. The mean age of the patient cohort was 34.1 ± 8.7 years. The mean expanded disability status scale (EDSS) score was 0.7 ± 1.4, and mean disease duration was 7.1 ± 3.5 years. The mean time interval between two MRI scans was 12.9 months (range 4–33 months).

MRI protocol

All brain MRIs were performed on two 3 T MRI scanners (Siemens Skyra, VE11A software version). The typical imaging protocol consisted of T1w and T1wGd (field of view = 24 cm, TR = 2300 ms, TE = 2.29 ms, TI = 900 ms, flip angle = 8°, voxel size = 1 × 1 × 1 mm3), T2w FLAIR (field of view = 24 cm, TR = 7600 ms, TE = 446 ms, TI = 2450 ms, flip angle = 120°, voxel size = 1 × 1 × 1 mm3), and multi-echo GRE for QSM (field of view = 24 cm, TR = 49 ms, TE1/ΔTE = 6.7/4.1 ms, number of TEs = 7, flip angle = 15°, acquisition matrix = 320 × 256, readout bandwidth = 260 Hz/pixel, slice thickness = 3 mm). QSM was constructed from GRE images with a fully automated zero-referenced Morphology Enabled Dipole Inversion (MEDI+0) method (Liu et al., 2017) that uses the ventricular cerebrospinal fluid (CSF) as a zero reference.

Automated lesion detection algorithm using T2w subtraction image

An automated image processing pipeline was developed for T2w lesion detection. Input images were first harmonized in a multi-step procedure based on FSL image analysis toolbox version 5.0.4 (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki) (Jenkinson et al., 2012). This consisted of brain extraction (using BET), bias field correction (using FAST), linear co-registration of T2w FLAIR images to a halfway space (using FLIRT), image intensity normalization (using FSLSTATS and FSLMATHS), and then subtraction of corrected, co-registered and normalized T2w FLAIR images (T2w subtraction, using FSLMATHS). For intensity normalization, the FSLSTATS-r command was applied to each T2w FLAIR image to compute the robust intensity range (2% and 98% percentiles, denoted as m and M, respectively) (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/Fslutils). Then, the FSLMATHS command was used to scale the image intensity of the second image I2 to match that of the first image I1: I2,scaled = αI2 + β, where α = (M1-m1)/(M2-m2) and β = ((M1-αM2) + (m1-αm2))/2. All other images (including T1wGd, T2w, and QSM) were also co-registered to the T2w FLAIR halfway space to facilitate image comparison by human readers.

Automatic lesion detection on T2w subtraction image was then carried out using an in-house statistical detection of changes (SDC) algorithm implemented in MATLAB, version R2016b. Briefly, SDC was formulated as a composite hypothesis test for the mean of the probability distribution of signal intensity on a T2w subtraction image (μ = 0 if the voxel is “unchanged” or μ ≠ 0 if “changed”). The SDC test statistic used the optimal Neyman-Pearson detector (Kay, 1998) over a 3-voxel local neighborhood. This SDC algorithm was designed according to the currently accepted minimum MS lesion size requirement of 3 mm (3 voxels in 1 mm3 isotropic images), and to the local connectivity constraint that the subtraction signals within this small neighborhood are similar. The threshold of the test statistic was then computed based on the desired false positive rate, PFP = 0.0001 in this study, selected because it provides the best detection power for a given PFP regardless of the unknown mean μ (uniformly most powerful detector) (Kay, 1998). The algorithm was programmed to generate color-coded boxes encompassing the detected lesions on T2w images to assist image review by the human reader (Fig. 1).

Fig. 1

MRI of representative cases with both new enhancing and new nonenhancing MS lesions. A, B: T2w FLAIR images at baseline and follow up date (the time interval between MRI time points in Case 1 and 2 is 4 and 18 months, respectively); C: T2w subtraction images; D: QSM maps; E: T1wGd images. The color-coded boxes represent the lesion masks detected by the SDC algorithm overlaid on the co-registered images to aid in rapid identification of potential new lesions. The new lesions within the pink box in Case 1 and green box in Case 2 are isointense on QSM, which is predictive of an enhancing lesion and is consistent with T1wGd. The lesions within the brown box in Case 1, and the yellow and blue boxes in Case 2 are hyperintense on QSM, which is predictive of nonenhancing lesions and is also consistent with T1wGd. The lesion shown in the pink box in Case 1 was missed by human readers, but correctly detected with the help of T2w + SDC.(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Data analysis

Detecting new Gd-enhancing lesions

Two neuroradiologists (A.G., 11 years of experience and S.Z, 6 years of experience) used T1w and T1wGd images to identify all enhancing lesions using conventional side-by-side comparison of images obtained at the two time points. The T2w subtraction images were not used in this initial assessment of lesion enhancement. Interpretations were made independently, with any disagreements resolved by consensus. The number and locations of these visually detected enhancing lesions were recorded.

Next, the neuroradiologists repeated the identification of new enhancing lesions using the color-coded locations of suspected new lesions delineated by T2w subtraction and SDC lesion detection algorithm (T2w + SDC), as well as all available images including T1w, T1wGd, T2w, and T2w subtraction.

Discriminating between new enhancing and new nonenhancing lesions

New lesions and their enhancement status (enhancing or nonenhancing) were identified using side-by-side comparisons of all available images by the human readers, assisted by color-coded locations of new lesions detected by the T2w + SDC algorithm. This lesion classification based on T1wGd and T2w + SDC (T2w + SDC + T1wGd) was used as the reference standard in the subsequent statistical analysis of diagnostic accuracy.

After a 4-week washout period to avoid recall bias, the neuroradiologists reviewed the previously identified new lesions on QSM images with T2w + SDC (T2w + SDC + QSM), and predicted the enhancement status of MS lesions while remaining blinded to the T1wGd images. Based on prior published work describing the time course of MS lesion susceptibility during the transition from enhancing to nonenhancing (Chen et al., 2014; Zhang et al., 2016; Zhang et al., 2016), we followed the principle that Gd-enhancing lesions tend to be isointense on QSM whereas nonenhancing lesions tend to be hyperintense on QSM. Lesions slightly visible on QSM with a QSM value less than that of CSF were predicted to be enhancing. The neuroradiologists made all assessments independently, with disagreements resolved by consensus.

Lesion ROIs were manually drawn on T2w images from the second time point and overlaid on QSM. When necessary, these ROIs were further edited to better match lesion geometry on QSM if lesions were visible on QSM. ROIs with similar sizes were also traced in lesion's adjacent NAWM for quantification of local susceptibility contrast. Veins or artifacts within the ROIs were removed manually.

Statistical analyses

The diagnostic accuracy of the proposed T2w + SDC + QSM method was computed using T2w + SDC + T1wGd as the reference standard. We also performed receiver operating characteristics (ROC) analysis based on a mixed effect logistic model to account for repeated measurements for each patient. The optimal susceptibility values were obtained from the corresponding optimal predicted probability cutoff to distinguish new enhancing lesions from new nonenhancing lesions. Separate ROC analyses were performed for minimum lesion sizes of 50 mm3 and 15 mm3 to determine the influence of lesion size on the accuracy of detection. The stringent 50 mm3 cutoff (approximately 4.6 mm in lesion diameter) was chosen based on the recommendation that lesion size should be at least 5 times larger than the slice thickness for lesion geometry to be captured reliably between scans (Firbank et al., 1999). The 15 mm3 cutoff was calculated assuming a spherical lesion shape with a diameter of 3 mm in accordance with the currently accepted minimum lesion size on MRI (Filippi et al., 2016). Statistical analyses were performed using R (version 3.2.0) for Windows.

Results

Without using T2w subtraction, the neuroradiologists identified 49 enhancing lesions in 33 patients. All of these 49 lesions were hyperintense on T2w subtraction images, indicating that lesion identification using T2w subtraction is 100% sensitive when compared with neuroradiologists' reading. With the aid of T2w + SDC, 54 new enhancing lesions were identified, including 5 enhancing lesions that were missed by the human readers without the use of T2w subtraction images (Fig. 1 shows an example of a lesion missed by human). A total of 111 new nonenhancing lesions were also identified with T2w + SDC. The average number of new enhancing and nonenhancing lesions per patient was 1.6 (range 1–7) and 3.5 (range 0–12), respectively.

Using T2w + SDC + QSM, the human readers found 65 new enhancing and 100 new nonenhancing lesions, which corresponds to a sensitivity of 90.7% (49/54) and specificity of 85.6% (95/111) for predicting new enhancing lesions (Table 1). For lesions ≥50 mm3, ROC analysis showed the area under the curve (AUC) to be 0.93 (95% CI, 0.87–0.99), with an optimal QSM cutoff value of 13.5 ppb resulting in a sensitivity of 88.4% and specificity of 88.6% in discriminating between new enhancing and new nonenhancing lesions. For lesions ≥15 mm3, the AUC was similar (0.93, 95% CI 0.89–0.97) with lower sensitivity of 77.9%, higher specificity of 94.0%, and an optimal QSM cutoff of 15.4 ppb (Fig. 2).

Table 1

Diagnostic accuracy of the proposed QSM-based no-Gd method for detecting new enhancing (+) and new nonenhancing (−) lesions with the conventional method based on T1wGd as the reference standard. In both methods, lesions were identified by two neuroradiologists with the assistance from T2w + SDC lesion detection algorithm.

		Reference Gd methodT2w + SDC + T1wGd
		Truly Gd+	Truly Gd-
Proposed no-Gd methodT2w + SDC + QSM	Predicted to be Gd+	49	16
	Predicted to be Gd-	5	95

Fig. 2

Receiver operating characteristic (ROC) curve analysis of MS lesion susceptibility values in discriminating between new enhancing and new nonenhancing MS lesions. A: ROC curve for lesions ≥50 mm3. The area under the curve (AUC) is 0.93 (95% CI, 0.87–0.99) with an optimal cutoff value of 13.5 ppb resulting in a sensitivity of 88.4% and specificity of 88.6%. B: ROC curve for lesions ≥15 mm3. The AUC is 0.93 (95% CI, 0.89–0.97), with an optimal cutoff value of 15.4 ppb resulting in a sensitivity of 77.9% and specificity of 94.0%.

Discussion

Our study showed that the proposed computer-assisted T2w + SDC method which combines T2w subtraction with SDC lesion detection algorithm is a highly sensitive technique for detecting new lesions, including potentially small or subtle enhancing lesions that may be missed by visual inspection of conventional MRI images alone. We also showed that T2w + SDC + QSM is highly accurate in discriminating between new enhancing and new nonenhancing MS lesions.

T2w subtraction images for lesion detection map the voxel-by-voxel signal change between the two imaging time points. Other investigations have explored the utility of T2w subtraction (Battaglini et al., 2014; Duan et al., 2008; Eichinger et al., 2017; Ganiler et al., 2014; Horsfield et al., 2016; Sweeney et al., 2013; van Heerden et al., 2015). In this study, we explored the use of an automated statistical detection of changes (SDC) algorithm which generates color-coded boxes on T2w subtraction and other co-registered source images to help the human readers quickly identify new lesions, particularly those with small volumes (20–35 mm3 for the 5 lesions missed by humans in our study). SDC combines the statistical optimality of the classic Neyman-Pearson detector with local voxel connectivity constraint to provide fast and robust detection of lesion changes on T2w subtraction images. In this study, SDC correctly detected 180 new lesions (median volume 42.5 mm3; volume range 6.6–3962 mm3) and missed only 2 lesions (13 and 34 mm3), demonstrating that it can provide sensitive detection of small lesions. In our preliminary testing, human detection of new lesions on whole brain T2w images with 1 mm3 isotropic resolution takes only an average of 2 min per case when assisted by SDC. A more rigorous comparison with conventional side-by-side image review and with other subtraction techniques will be considered in future work. While the proposed T2w + SDC algorithm can detect both lesion volume increase and decrease over time, we have focused on new lesions (volume increase) in this study. The presence of new enhancing lesions has been widely recognized as indicative of new disease activity and can directly impact MS clinical decision making (Sommer et al., 2018; Yamamura et al., 2017).

In this study, all subjects were imaged on two Siemens Skyra 3T scanners with very similar hardware and software, which allowed high quality T2w subtraction. However, some challenges to optimal subtraction can occur when using two different scanners, as may be the case in clinical practice. Generally speaking, images obtained at the same site can be expected to be acquired with similar pulse sequences and imaging parameters. However, different scanners, especially those from different manufacturers or at different field strengths, may produce images with slightly different brain contrasts. In such a scenario, it may be necessary to perform image normalization and subtraction on GM and WM tissues separately to ensure uniformly high quality subtraction across the brain. The use of different RF coils is likely to have minimal impact on the subtraction, because bias field correction performed during post-processing removes RF-related signal inhomogeneity. Our encouraging results warrant further studies to evaluate the effectiveness of T2w subtraction for detecting lesions on images acquired using different scanners. Another factor that may influence the quality of T2w subtraction is the time interval between follow-up scans. As the scanning interval becomes longer, changes in brain anatomy due to gray and white matter atrophy or ventricular hypertrophy can be more pronounced, thus potentially interfering with T2w subtraction (Ganiler et al., 2014). These are important considerations and warrant further evaluation.

QSM-based MRI studies have shown predictable longitudinal changes in susceptibility of MS lesions, in which susceptibility increases as lesions evolve from active/acute to chronic over time (Chen et al., 2014). Our current results suggest that the proposed T2w + SDC + QSM approach can accurately predict BBB integrity (enhancement status) without the need for Gd. This imaging strategy can also provide insight into the smoldering microglial inflammation that cannot be characterized by Gd enhancement. Given mounting concerns about the safety of Gd deposition in the brain (Kanda et al., 2017), the results of this and other studies (Gupta et al., 2017; Idee et al., 2008) suggest that long established MS imaging protocols may need to be re-evaluated. Specifically, non-contrast MRI protocols leveraging QSM and T2w subtraction techniques should be further investigated as an alternative imaging strategy that would not expose MS patients to the known and unknown risks of repetitive Gd injections. In the current imaging workflow implemented at our institution, QSM data is acquired using a product multi-echo 3D fast gradient echo sequence (about 5 min) which is readily available on modern scanners. After finishing the QSM series, images are automatically transferred to a dedicated computer for online QSM reconstruction and QSM images (in DICOM format) are automatically sent back to the MRI scanner image database for archiving (2 min). Clinical QSM is therefore feasible and will likely gain wider acceptance in routine MRI surveillance of MS.

Our work builds upon recent studies showing that QSM is effective in discriminating new Gd-enhancing lesions from new nonenhancing lesions (Zhang et al., 2016; Zhang et al., 2016). Unlike these prior studies, we integrated the T2w subtraction image with the SDC lesion detection algorithm and QSM into the image interpretation pipeline. This approach combines a highly sensitive screen for small lesions with specific insights into lesion inflammation provided by QSM. In addition, through the integration of T2w subtraction mapping with automatic lesion detection, the non-contrast MRI protocol used in our study is readily translatable to the clinical workflow. Overlaying new lesion masks identified from the T2w subtraction images can allow for rapid and simple characterization of lesion features on QSM. Such co-registration of T2w and QSM images is important for confident qualitative and quantitative (ROI-based) assessment of a lesion's QSM signal intensity. Finally, because we relied on 1 mm3 isotropic T2w for subtraction mapping (unlike prior work which used a slice thickness of 3 mm (Zhang et al., 2016)), we were able to investigate the differences in diagnostic accuracy stratified by minimum detectable lesion size. Our results suggest that the T2w + SDC + QSM approach achieves high overall accuracy for detecting lesions as small as 15 mm3, which corresponds to a spherical lesion with a diameter of 3 mm, the currently accepted minimum linear dimension recommended in the most recent guidelines of MRI criteria for MS diagnosis (Filippi et al., 2016).

There are some limitations in this study, including its sample size and retrospective design. Nonetheless, we believe that our results warrant prospective testing of this no-Gd protocol in a larger patient cohort. Furthermore, we dichotomized lesion enhancement status (present versus absent) without detailed analysis of enhancement morphology or volume. Though we believe that our current approach is similar to that of clinical practice where the presence or absence of Gd-enhancing lesions is the most critical information extracted from T1wGd images, future studies should investigate how enhancement volume and morphology influence diagnostic accuracy. Another limitation is the exclusion of patients without brain lesions in the assessment of diagnostic accuracy. It is rare that patients with confirmed MS do not have brain lesions; a study with a larger sample size may be needed to adequately sample this cohort. Finally, we only considered WM lesions in our study to circumvent the limited contrast of GM lesions on QSM, as cortical and deep GM typically have high QSM values that are similar to that of a lesion. Further work to assess the utility of QSM in detecting GM lesions is warranted.

In conclusion, T2w + SDC + QSM is highly accurate in detecting new white matter MS lesions and discriminating new enhancing from new nonenhancing MS lesions. Our results support the use of T2w + SDC + QSM for routine surveillance of MS patients without the risks of repetitive Gd injections.

Declarations of interest

None