Application of the 2017 McDonald criteria in a Chinese population with clinically isolated syndrome.

BACKGROUND: Diagnostic criteria for multiple sclerosis have evolved over time, with the most recent being the 2017 McDonald criteria. Evidence is lacking regarding the validity of the 2017 McDonald criteria among the Asian population. Therefore, this study aims to evaluate the diagnostic performance of the 2017 McDonald criteria in Chinese patients with clinically isolated syndrome (CIS).
METHODS: A total of 93 patients with initial findings suggestive of CIS in a tertiary hospital in China from 2012 to 2017 were included in this retrospective study. Baseline and follow-up data were reviewed. Diagnostic performance (sensitivity, specificity, accuracy), was assessed and survival analysis was performed for the 2017 and 2010 McDonald criteria respectively.
RESULTS: Among the 93 Chinese patients with CIS, 57 were female (61.3%) and the median (interquartile range) age of onset was 37 (31.3-41.8) years. The 2017 McDonald criteria displayed a higher sensitivity (75.0% versus 14.6%, p < 0.0001), lower specificity (47.1% versus 100.0%, p < 0.05) but an overall higher accuracy (67.7% versus 36.9%, p < 0.0001) when compared with the 2010 iteration. The novel criteria allow for a better detection of MS at baseline (40.8% versus 9.9%, p < 0.0001).
CONCLUSION: The 2017 McDonald criteria had a higher sensitivity but lower specificity than the 2010 iteration. Overall it facilitated an earlier and more accurate diagnosis of multiple sclerosis in Chinese patients with CIS.

Introduction

Multiple sclerosis (MS) is a debilitating inflammatory central nervous system (CNS) disorder, typically with a relapsing-remitting disease course. Establishment of the diagnosis at the first demyelinating attack, termed clinically isolated syndrome (CIS), remains critical in terms of patient prognosis.[1] Important as it is, the diagnostic process for MS has never been straightforward.[2] Given the difficulty in defining the disease, multiple revisions have been made to its diagnostic criteria. Compared with the 2010 version,[3] the latest 2017 McDonald criteria for MS diagnosis made 3 main changes:[4] inclusion of cortical lesions as evidence of dissemination in space (DIS), abolishment of differentiation between symptomatic and asymptomatic lesions, and inclusion of cerebrospinal fluid (CSF)-specific oligoclonal bands (OCBs) as evidence of dissemination in time (DIT).

In the Asian population, however, it remains unknown whether the 2017 McDonald criteria remains applicable.[5,6] Given the diversity in characteristics of MS across ethnicities, nations and regions, the 2017 position paper also listed validation in different populations as 1 of its high-priority areas for research.[7] Despite significant improvements, evidence is still lacking on the validity of the 2017 McDonald criteria in China, limiting its application among the Chinese.

Therefore, we set to evaluate the diagnostic performance of the 2017 McDonald criteria. A group of 93 Chinese patients diagnosed with CIS was selected for the evaluation. Our study offered the first evidence for the application of the 2017 McDonald criteria in China, validating its use in Chinese CIS patients.

Methods

Study participants

Individuals with an initial demyelinating attack suspected of MS were recruited between 2012 and 2017 in the Second Affiliated Hospital School of Medicine Zhejiang University, a tertiary hospital in China (Figure 1). The inclusion criteria were as follows: (a) age of onset between 11 and 60 years old; (b) an initial clinical attack suggestive of CIS, with both patient-reported symptoms and objective findings; (c) baseline brain magnetic resonance imaging (MRI) (containing gadolinium enhancement) obtained within 3 months of the initial symptom onset; (d) followed up for at least 2 years or diagnosed with clinically definite MS (CDMS); (e) follow-up brain MRI available within 24 months of symptom onset; (f) had no disease-modifying treatment (DMT) before the second clinical episode. Exclusion criteria were as follows: (a) primary progressive subtype of MS (b) aquaporin-4 or myelin oligodendrocyte glycoprotein positivity in the CIS attack or during follow-up. The definition for CIS, relapses and CDMS were defined previously in the literature.[8,9]

Figure 1.

Flow chart for patient selection.

AQP4, aquaporin-4; DMT, disease-modifying treatment; MRI, magnetic resonance imaging; PPMS, primary progressive multiple sclerosis.

Study design

We retrospectively reviewed the medical records and radiological imaging of all enrolled patients. Patients were assessed at baseline and follow-up. At baseline, brain MRI (containing gadolinium enhancement) and routine laboratory tests were performed to rule out alternative diagnosis. At follow-up, brain MRI was ordered when relapses were suspected. We collected information on sex, age of onset, presenting phenotypes [monofocal onset (optic neuritis, myelopathy, brainstem or cerebellar syndrome, hemispheric syndrome), multifocal onset] and OCB results. Clinical outcome was defined as CDMS until the last follow-up. Patients were divided into CDMS converters and non-converters accordingly.

All patients (n = 93) were included for DIS assessment and survival analysis. Patients with available OCB results at baseline (n = 65) were selected for the assessment of DIT and the full criteria (DIS+DIT).

Our study was approved by the ethics committee of the Second Affiliated Hospital School of Medicine Zhejiang University (approval number: 2019-082). Consent was obtained from all patients participating in this study for the use of their anonymized MRI examinations and clinical details for research purposes.

Procedures

MRI scans were performed with a GE 1.5-T MRI scanner (Siemens Healthcare, Erlangen, Germany). MRI were obtained in the axial plane. The following sequences were used: T1-weighted images, T2-weighted images, fluid-attenuated inversion recovery images and T1-weighted images with gadolinium administration. CSF analysis for OCBs was performed using isoelectric focusing. OCB status was considered positive if there were at least 2 unique bands in CSF compared with serum.[10]

Statistical analysis

For statistical analysis and graphs, we used R (Version 3.3.3 for Mac). True positives (TP), false positives (FP), true negatives (TN) and false negatives (FN) were defined as previously described in the literature.[11]

To evaluate the diagnostic performance, sensitivity, specificity, and accuracy were calculated as previously described.[11] Bias-corrected and accelerated bootstrap method was used to estimate 95% confidence intervals.[12,13] Comparison of sensitivity and specificity was done with the McNemar test (when b + c < 25, exact McNemar test was used).

Comparison of continuous parametric variables was calculated using a two-tailed t test. Comparison of nonparametric data was calculated using with Mann–Whitney U test. For categorical data, we used χ2. To evaluate time to CDMS with different diagnostic criteria, Kaplan–Meier curves were assessed with log-rank tests. Multivariate cox regression was performed to calculate adjusted hazard ratios (HRs) using time to CDMS diagnosis as the outcome, adjusted for age (continuous), sex (binary), the presenting phenotype (categorical), and the availability of OCBs (binary). Patients who did not meet the criteria for CDMS diagnosis or with a confirmed alternative diagnosis were considered censored observations. P values < 0.05 were considered statistically significant and all p values were two-sided.

Results

Patient characteristics

In total, 93 patients with CIS were recruited. Demographic features of the recruited patients are summarized in Table 1. Of these 93 patients, 71 (76.3%) were diagnosed with CDMS during follow-up. The remaining 22 (23.7%) patients failed to reach the CDMS diagnosis at the last follow-up [median follow-up time 44 months, interquartile range (IQR) 29–62.5 months], of whom 17 did not develop a second demyelinating event and 5 were confirmed with an alternative diagnosis (CNS vasculitis, cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy, hereditary leukodystrophy, CNS lymphoma and stiff person syndrome).

Table 1.

Baseline characteristics.

Demographic	n = 93
Female	57 (61.3%)
Age at onset, median (IQR), year	37 (31.3–41.8)
Clinical syndromes	n = 93
Optic neuritis	12 (15.4%)
Myelopathy	32 (41.0%)
Brainstem or cerebellar syndrome	28 (35.9%)
Hemispheric syndrome	6 (7.7%)
Multifocal	15 (16.1%)
MRI evaluation	n = 93
Disease duration at baseline brain MRI, median (IQR), month	0.5 (0–1)
Enhancement sequence of spinal MRI	39/60 (65.0%)
CSF analysis	n = 65
Disease duration at baseline lumbar puncture, median (IQR), month	0.83 (0–2)
Positive OCBs	36 (55.4%)
Positive OCBs in CDMS converter: non-converter	30/48 (62.5%) : 6/17 (35.3%), p = 0.10
Outcome	n = 93
CDMS at follow-up	71 (76.3%)
Time to CDMS, median (IQR), month	10 (3.0–24.0)
Follow-up duration in non-converters, median (IQR), month	44 (29.0–62.5)

CDMS, clinically definite multiple sclerosis; CSF, cerebrospinal fluid; IQR, interquartile range; MRI, magnetic resonance imaging; OCB, oligoclonal band.

Dissemination in space and time

For DIS analysis, all recruited patients were included (n = 93), among which 71 (76.3%) developed CDMS. The 2010 DIS criteria were fulfilled in 48 (51.6%) patients, of which 38 (79.2%) developed CDMS. An additional 29 (31.2%) patients fulfilled the 2017 DIS criteria, of which 23 (79.3%) developed CDMS. The 2017 DIS criteria (85.9%) showed a significantly higher sensitivity than the 2010 version (53.5%) (p < 0.0001). By contrast, specificity of the 2017 DIS criteria was also lower than the 2010 version (2017 versus 2010: 27.3% versus 54.5%), though statistical significance was not reached (p = 0.06). In addition, the 2017 DIS criteria showed a significantly higher accuracy than the 2010 version (2017 versus 2010: 72.0% versus 53.8%, p < 0.0001).

For analysis of DIT, only patients with OCB results at baseline were recruited (n = 65), among whom 48 (73.8%) developed CDMS. The 2010 DIT criteria were fulfilled in 10 (15.4%) patients, of which 9 (90%) were later diagnosed with CDMS. An additional 37 (56.9%) patients met the 2017 DIT criteria, among which 29 (78.4%) developed CDMS. The 2017 DIT criteria were far more sensitive than the 2010 DIT criteria (2017 versus 2010: 79.2% versus 18.8%, p < 0.0001), less specific (2017 versus 2010: 47.1% versus 94.1%, p < 0.05), and more accurate (2017 versus 2010: 70.8% versus 38.5%, p < 0.0001). Data are summarized in Table 2.

Table 2.

Diagnostic performance of the 2010 and 2017 McDonald criteria for multiple sclerosis.

	TP (n)	FP (n)	FN (n)	TN (n)	Sensitivity	Specificity	Accuracy
DIS
2010	38	9	33	12	53.5 (53.3–54.4)	54.5 (53.4–55.6)	53.8 (53.4–54.5)
2017	61	15	10	6	85.9 (85.6–86.6)	27.3 (26.4–28.6)	72.0 (71.6–72.7)
DIT
2010	12	3	43	18	18.8 (18.4–19.7)	94.1 (93.5–94.7)	38.5 (38.3–39.7)
2017	38	8	10	8	79.2 (78.5–79.6)	47.1 (46.7–49.3)	70.8 (70.4–71.5)
DIS + DIT
2010	9	0	62	21	14.6 (13.9–14.9)	100.0	36.9 (35.8–37.1)
2017	36	8	12	8	75.0 (74.2–75.6)	47.1 (46.7–49.5)	67.7 (67.2–68.6)

Values given within parentheses represent 95% confidential intervals.

DIS, dissemination in space; DIT, dissemination in time; FN, false negative; FP, false positive; TP, true positive; TN, true negative.

The 2010 McDonald criteria versus 2017 McDonald criteria 2017

To evaluate the diagnostic performance of the McDonald criteria, we selected patients who have undergone OCB testing for further analysis (n = 65). Seven (10.8%) patients fulfilled the 2010 McDonald criteria, all of which developed CDMS. By contrast, an additional 38 (58.5%) patients met the 2017 full criteria, and 29 (76.3%) were diagnosed with CDMS. The 2017 McDonald criteria were far more sensitive (2017 versus 2010: 75.0% versus 14.6%, p < 0.0001) but less specific (2017 versus 2010: 47.1% versus 100.0%, p < 0.05) in the Chinese population. The overall accuracy was higher for the 2017 criteria compared with the 2010 version (2017 versus 2010: 67.7% versus 36.9%, p < 0.0001). Nine patients that caused the loss in specificity were identified (fulfilling the 2017 criteria but not the 2010 criteria at baseline and failed to develop CDMS). Three were later diagnosed with an alternative diagnosis (CNS vasculitis, hereditary leukodystrophy, lymphoma), of which 1 had positive OCBs. The other 6 remained relapse-free without a definite diagnosis since the first event, of which 5 had positive OCBs at baseline.

All recruited patients were included in survival analysis (n = 93). MS can be diagnosed much earlier using the 2017 McDonald criteria than the 2010 criteria (χ2 = 32.06, p < 0.0001). Of 71 patients diagnosed with CDMS, 7 fulfilled the 2010 McDonald criteria within 1 year, 7 within 3 years and 9 within 5 years. By contrast, the 2017 criteria led to the identification of 29 CDMS patients within 1 year, 36 within 3 years and 41 within 5 years. The Kaplan–Meier survival curves for CDMS diagnosis, the 2010 McDonald criteria and the 2017 McDonald criteria for CIS patients are presented in Figure 2. Adjusted HRs (adjusted for age, sex, presenting phenotype and presence of OCBs) through multivariate Cox analysis for each criteria at baseline are summarized in Table 3.

Figure 2.

Time from CIS to MS diagnosis.

Survival curves from CIS to MS diagnosis according to the 2010 McDonald criteria, 2017 McDonald criteria, and CDMS.

CIS, clinically isolated syndromes; MS, multiple sclerosis.

Table 3.

Adjusted hazard rations and comparisons of the 2017 McDonald criteria with the 2010 McDonald criteria.

	Adjusted hazard ratios	p value
DIS only
2010 criteria	1.14 (0.70–1.85)	0.5955
2017 criteria	1.46 (0.73–2.89)	0.2828
DIT only
2010 criteria	1.75 (0.88–3.48)	0.1100
2017 criteria	1.19 (0.68–2.09)	0.5368
DIS plus DIT
2010 criteria	2.77 (1.21–6.35)	0.0162[*]
2017 criteria	1.31 (0.77–2.22)	0.3185

Values given within parentheses represent 95% confidential intervals.

Adjusted hazard ratios were obtained from Cox regression models using time to clinically definite multiple sclerosis as the outcome (adjusted for age, sex, the presenting phenotype and availability of oligoclonal bands or IgG index).

Significance codes: *p < 0.05.

DIS, dissemination in space; DIT, dissemination in time.

Contribution of OCBs and symptomatic enhancing lesions

Only patients having OCB (n = 65) results at baseline were included for the following analysis. Among the extra 38 patients diagnosed by the 2017 McDonald criteria but overlooked by the 2010 iteration, 27 (71.1%) could be attributed to CSF OCB positivity, among which 9 (33.3%) failed to develop CDMS during follow-up. Inclusion of OCBs increased the sensitivity (2017 versus 2010: 75.0% versus 31.3%, p < 0.0001) but lowered the specificity (2017 versus 2010: 47.1% versus 88.2%, p < 0.05) of the McDonald criteria. Another major change of the 2017 criteria was the inclusion of symptomatic lesions as part of DIT and DIS. Of those additionally diagnosed by the 2017 criteria, 16 (42.1%) could be due to symptomatic lesions, among which 4 (25.0%) did not develop CDMS. We found that the inclusion of symptomatic lesions increased the sensitivity (2017 versus 2010: 31.3% versus 14.6%, p < 0.05) without significantly lowering the specificity (2017 versus 2010: 88.2% versus 100%, p = 0.5). Furthermore, both the inclusion of OCBs and symptomatic lesions allowed a much earlier diagnosis of MS according to the survival analysis (Figure 2). Adjusted HRs are displayed in Table 3.

Discussion

Validation of the 2017 McDonald criteria in diverse populations, including the Chinese, remains an area of concern.[4,5,14] Our study, in a real-world setting in a Chinese population, revealed that the 2017 McDonald criteria was more sensitive (75.0% versus 14.6%, p < 0.0001) than the 2010 criteria despite a decrease in specificity (47.1% versus 100%, p < 0.05). Overall the 2017 McDonald criteria was more accurate than that of the 2010 criteria (67.7% versus 36.9%, p < 0.0001). In addition, with the novel criteria more MS diagnosis could also be made at baseline (40.8% versus 9.9%, p < 0.0001).

This present study, to our knowledge, provides the first evidence for the validity of the 2017 McDonald criteria in the Chinese population. A higher sensitivity lowers the hurdle for reaching MS diagnosis and allows an earlier DMT treatment, while specificity is important for correct diagnosis and appropriate management. Previous studies, in line with ours, have reached a consensus on the higher sensitivity of the 2017 criteria (ranging from 66% to 100%).[11,15-18] In addition, our study also found the specificity of the 2017 criteria (47.1%) to be significantly lower than the 2010 version, consistent with results from other cohorts (showing a specificity from 13.8% to 85%).[4,11,15-17] This result may imply that adoption of the 2017 criteria, though able to identify more patients at baseline, could culminate in misdiagnosis and overdiagnosis. However, the actual specificity of the 2017 criteria may be higher than estimated. Reasons include the limited follow-up, the use of a strict definition of CDMS as the endpoint, and the lack of advanced MRI protocols for patient evaluation. Among the 9 patients causing the loss in specificity, three were confirmed with an alternative diagnosis. The other 6 remained relapse-free within a median follow-up of 36 months. It is possible that those non-converters had an insidious MRI progression or new cortical lesions that were not detected by our radiological protocol. Given their typical MS-like presentations and OCB positivity at baseline, the 6 non-converters have a high chance to develop a second relapse with a longer follow-up.[19] Taken together, we advocate the use of 2017 McDonald criteria in the Chinese population based on the higher sensitivity, accuracy and its ability to identify more MS patients at an early stage of the disease course.

We also examined the contribution of the 2 major revisions in the 2017 McDonald criteria. The inclusion of CSF-OCBs and symptomatic lesions were both associated with a higher sensitivity for MS diagnosis, in parallel with previous studies.[4,13] Notably, inclusion of CSF-OCBs significantly reduced the specificity. OCB was positive in around half of patients with CIS, similar to other studies on the Chinese population.[20] It was positive in 62.5% of CDMS patients and in 35.3% of non-converters. Among the 5 patients later confirmed with an alternative diagnosis, 60% had OCB positivity at baseline. The data brings to question the specificity of the OCB testing for MS diagnosis.[14] As a matter of fact, the predictive value of OCBs alone for CDMS conversion remains inconclusive.[21,22] Previous studies, however, have consistently revealed that OCBs, when combined with DIS, were highly predictive of relapses.[15] In addition, incorporating the symptomatic lesions in the DIS criteria numerically lowered the specificity as well, though statistical significance was not reached. We attribute the trend of a lower specificity to the limited follow-up and the sample size. Other studies also found an increase of false positives after including symptomatic lesions into the criteria. They also found that, with a longer follow-up or the addition of DIT information, specificity improves.[23,24] Despite the numerical decrease in specificity, the inclusion of symptomatic lesions simplified the diagnostic process and facilitated its implementation. Taken together, both the inclusion of OCBs and symptomatic lesions contributed to the increased sensitivity, while OCBs attributed more to a lower specificity of the 2017 criteria.

Our study has several limitations. First, not all patients underwent a baseline lumbar puncture and spinal cord MRI. This consideration was based on the scope of our study to reflect the real-world scenario for patients diagnosed with CIS. Both of the investigations were not mandatory and the decision was left to the discretion of the treating physician according to the 2017 position paper.[4,5] Second, given the limited accessibility to advanced MRI techniques, consistent with the real-world setting, the use of 1.5T MRI impeded our ability to detect cortical lesions. The failure to identify cortical lesions as a representation of DIS may contribute to the lower estimated specificity in our study. Future studies with high resolution MRIs with 3.0T would aid in a more accurate evaluation of the 2017 criteria.

Conclusion

This study, based on a real-world setting, offered the first evidence validating the 2017 McDonald criteria among the Chinese population. The 2017 revision had a higher sensitivity, but lower specificity compared with the 2010 iteration, allowing more patients to be diagnosed at an earlier stage in the disease.