Performance of the proposed ACR-EULAR classification criteria for systemic lupus erythematosus (SLE) in a cohort of patients with SLE with neuropsychiatric symptoms.

Recently, new American College of Rheumatology–EULAR classification criteria for systemic lupus erythematosus (SLE) have been proposed.

In our cohort of patients with (suspected) SLE with neuropsychiatric (NP) symptoms, sensitivity of the proposed criteria was high, but specificity was suboptimal.

Sensitivity further improved by including patients with antinuclear antibody–negative lupus nephritis (LN) and specifying the neurological domain.

By further improving sensitivity as suggested, more patients with NPSLE and LN will be able to participate in future clinical SLE trials.

Introduction

Systemic lupus erythematosus (SLE) is a heterogenic connective tissue disease with a broad spectrum of clinical and laboratory manifestations. Due to this heterogeneity, SLE remains challenging to diagnose in clinical practice. In order to create a more homogeneous patient group, the American College of Rheumatology (ACR) developed classification criteria for research purposes in 1972.1 These criteria were revised in 1982 and 1997.2 3

In 2012, the Systemic Lupus International Collaborating Clinics (SLICC) developed and validated new SLE classification criteria.4 The SLICC 2012 criteria performed better than the revised 1997 ACR criteria in terms of sensitivity (97% vs 83%), but were less specific (84% vs 96%).4

Recently, new ACR–EULAR criteria have been proposed in order to improve specificity, while keeping the optimal sensitivity of the SLICC 2012 criteria. Several new elements were added, including the presence of antinuclear antibodies (ANA) as entry criterium, weighted scores for each criterium and domain scores.5–7 It is currently unknown how these criteria perform in patients with SLE with neuropsychiatric (NP) symptoms, one of the least understood manifestations of SLE.

Therefore, we aimed to evaluate the performance of the proposed ACR–EULAR criteria in our cohort of patients with SLE presenting with NP symptoms.

Methods

A retrospective cohort study was performed using electronic medical records of patients referred to the NPSLE clinic in the Leiden University Medical Center (LUMC). Information regarding the proposed ACR–EULAR criteria, the SLICC 2012 criteria and the 1997 ACR criteria was collected. Items of the different classification criteria that were attributed to other causes, for example, medication, were not counted. All patients underwent standardised multidisciplinary assessment, as described previously.8 In case of definitive NPSLE diagnosis, the applicable 1999 ACR NPSLE syndromes were assigned.

In the LUMC, ANA is detected using immunofluorescence on immobilised HEp-2 cells (Biomedical Diagnostics). ANA is considered positive at a titre of ≥1:40.

Sensitivity, specificity and accuracy were calculated for the proposed ACR–EULAR criteria, the SLICC 2012 criteria and the 1997 ACR criteria, using the clinical diagnosis as golden standard. Statistical analysis was performed using SPSS for Windows V.23.0. Also, 95% CIs were calculated using the Clopper-Pearson CIs.

Results

A total of 360 patients were included, of which 294 (82%) had the clinical diagnosis of SLE. Mean age was 43 years and the majority of patients were women (86%). Baseline characteristics are presented in table 1.

Table 1

Prevalence of different symptoms in patients with and without clinically diagnosed SLE

	SLE (n=294)	No SLE (n=66)
Age, years (mean±SD)	43±14	46±15
Gender (female, %)	256 (87.1)	55 (83.3)
ACR 1997 criteria (n, %)
Malar rash	129 (43.9)	9 (13.6)
Discoid rash	52 (17.7)	7 (10.6)
Photosensitivity	155 (52.7)	22 (33.3)
Oral ulcers	126 (42.9)	19 (28.8)
Non-erosive arthritis	183 (62.2)	11 (16.7)
Pleuritis or pericarditis	76 (25.9)	2 (3.0)
Renal disorder	88 (29.9)	3 (4.5)
Neurological disorder	37 (12.6)	6 (9.1)
Haematological disorder	145 (49.3)	10 (15.2)
Immunological disorder	221 (75.2)	20 (30.3)
Positive ANA	283 (96.3)	50 (75.8)
SLICC 2012 criteria (n, %)
Acute or subacute cutaneous lupus	148 (50.3)	12 (18.2)
Chronic cutaneous lupus	52 (17.7)	7 (10.6)
Oral ulcers	126 (42.9)	19 (28.8)
Non-scarring alopecia	44 (15.0)	2 (3.0)
Arthritis	183 (62.2)	11 (16.7)
Serositis	76 (25.9)	2 (1.0)
Renal	88 (29.9)	3 (4.5)
Neurological Acute confusional state/delirium	9 (2.7)	0 (0.0)
Psychosis	12 (4.1)	2 (3.0)
Seizure	28 (9.5)	2 (3.0)
Mononeuritis	5 (1.7)	1 (1.5)
Myelitis	9 (3.1)	2 (3.0)
Neuropathy	10 (3.4)	4 (6.1)
Leucopenia	140 (47.6)	15 (22.7)
Thrombocytopenia	58 (19.7)	4 (6.1)
Immunological criteria ANA	283 (96.3)	50 (75.8)
Anti-Sm	36 (12.2)	2 (3.0)
Antiphospholipid antibody (IgG, IgM, LAC, anti-β2 glycoprotein)	143 (48.6)	17 (25.8)
Low complement (C3, C4)	168 (57.1)	14 (21.2)
Direct Coombs test	21 (7.1)	0 (0.0)
Proposed ACR–EULAR criteria
Constitutional domain Fever	71 (24.2)	11 (16.7)
Mucocutaneous domain Non-scarring alopecia	44 (15.0)	2 (3.0)
Oral ulcers	126 (42.9)	19 (28.8)
Subacute cutaneous/discoid lupus	94 (32.0)	17 (25.8)
Acute cutaneous lupus	148 (50.3)	12 (18.2)
Musculoskeletal domain Arthritis	183 (62.2)	11 (16.7)
Serositis domain Pleural or pericardial effusion	58 (19.8)	1 (1.5)
Acute pericarditis	34 (11.6)	1 (1.5)
Haematological domain Leucopenia	118 (40.3)	10 (15.2)
Thrombocytopenia	58 (19.7)	4 (6.1)
Autoimmune haemolysis	43 (14.6)	1 (1.5)
Renal domain Proteinuria	66 (22.4)	1 (1.5)
Lupus nephritis class II/V	31 (10.5)	2 (3.0)
Lupus nephritis class III/IV	37 (12.6)	0 (0.0)
Complement protein domain Low C3 or low C4	168 (57.3)	14 (21.5)
Low C3 and low C4	101 (34.5)	2 (3.1)
Highly specific antibodies domain Anti-dsDNA antibody	169 (57.7)	15 (22.7)
Anti-Smith antibody	36 (12.2)	2 (3.0)
Antiphospholipid antibodies domain IgG, IgM, β2 glycoproteins, lupus anticoagulant	143 (48.6)	17 (25.8)
Neurological domain Acute confusional state/delirium	9 (2.7)	0 (0.0)
Psychosis	12 (4.1)	2 (3.0)
Seizure	28 (9.5)	2 (3.0)
1999 ACR neuropsychiatric syndromes*
Aseptic meningitis	0 (0.0)
Cerebrovascular disease	30 (10.2)	–
Demyelinating syndrome	0 (0.0)	–
Headache	12 (4.1)	–
Movement disorder (chorea)	3 (1.0)	–
Myelopathy	6 (2.0)	–
Seizure disorders	11 (3.7)	–
Acute confusional state	6 (2.0)	–
Anxiety disorder	2 (0.7)	–
Cognitive dysfunction	31 (10.5)	–
Mood disorder	9 (3.1)	–
Psychosis	7 (2.4)	–
Acute inflammatory demyelinating polyradiculoneuropathy	0 (0.0)	–
Autonomic disorder	0 (0.0)	–
Mononeuropathy	0 (0.0)	–
Myasthenia gravis	0 (0.0)	–
Neuropathy, cranial	0 (0.0)	–
Plexopathy	0 (0.0)	–
Polyneuropathy	3 (1.0)	–

*The 1999 ACR NPSLE syndromes were assigned after the presence of NPSLE was confirmed during multidisciplinary assessment.

ACR, American College of Rheumatology; ANA, antinuclear antibody; NP, neuropsychiatric; SLE, systemic lupus erythematosus; SLICC, Systemic Lupus International Collaborating Clinics.

Of the 66 patients without the clinical diagnosis of SLE, 20 patients had SLE-like disease, 12 patients undifferentiated connective tissue disease, 11 primary Sjögren’s syndrome and 8 patients had mixed connective tissue disease. The diagnoses of the remaining 15 patients were chilblain LE, chronic discoid lupus, subacute cutaneous lupus, antiphospholipid syndrome, dermatomyositis, Calcinosis-Raynaud phenomenon-esophageal involvement-sclerodactyly-telangiectasia (CREST) syndrome, juvenile idiopathic arthritis, Behçet-like disease and somatoform disorder.

Of the 294 patients with the clinical diagnosis SLE, 257 (87%) fulfilled the proposed ACR–EULAR criteria, 249 (85%) patients fulfilled the 2012 SLICC criteria and 261 (89%) patients fulfilled the 1997 ACR criteria.

The sensitivity, specificity and accuracy of the different criteria are presented in table 2. The proposed ACR–EULAR criteria showed a sensitivity of 87% (95% CI 83% to 91%), specificity of 74% (95% CI 62% to 84%) and an accuracy of 85% (95% CI 81% to 89%). The 2012 SLICC criteria had a sensitivity of 85% (95% CI 80% to 89%), a specificity of 76% (95% CI 64% to 85%) and an accuracy of 83% (95% CI 79% to 87%). The 1997 ACR criteria had a sensitivity of 89% (95% CI 85% to 92%), a specificity of 89% (95% CI 80% to 96%) and an accuracy of 89% (95% CI 85% to 92%).

Table 2

Sensitivity, specificity and accuracy of the ACR 1997, proposed ACR–EULAR criteria and SLICC 2012 criteria

	ACR 1997	ProposedACR–EULAR criteria	SLICC 2012
Sensitivity (95% CI)	89% (85% to 92%)	87% (83% to 91%)	85% (80% to 89%)
Specificity (95% CI)	89% (80% to 96%)	74% (62% to 84%)	76% (64% to 85%)
Accuracy (95% CI)	89% (85% to 92%)	85% (81% to 89%)	83% (79% to 87%)

ACR, American College of Rheumatology; SLICC, Systemic Lupus International Collaborating Clinics.

Discussion

We investigated the performance of the proposed ACR–EULAR criteria for SLE in our cohort of patients with (suspected) SLE and NP symptoms and demonstrated that sensitivity was high, but specificity was suboptimal. Our finding contrasts the results of Aringer et al.6, who found that the proposed criteria had a sensitivity close to the SLICC 2012 criteria, while specificity was similar to the ACR 1997 criteria.

There are several possible explanations for this difference. The first explanation relates to the design of our cohort, which is a selected population of patients with (suspected) SLE and NP symptoms. Patients are generally referred to our clinic when diagnostic difficulties arise, which means that patients without the clinical diagnosis of SLE often have SLE-mimicking syndromes. Specificity might therefore be lower than expected. This is demonstrated for example by the high prevalence of anti-dsDNA positivity in patients without the clinical diagnosis of SLE (23%), leading to a low sensitivity (57%) and specificity (72%) of this test in our cohort. We expect that this is due to a strong referral bias: patients with autoimmune phenomena and a positive anti-dsDNA are more likely to be referred to our NPSLE clinic in case of NP symptoms than patients with a negative anti-dsDNA. Furthermore, due to the retrospective design of this study, some criteria were difficult to retrieve. This could have led to an underestimation of, for example, the constitutional domain, thereby leading to a lower sensitivity of the proposed criteria in our cohort than truly present.

The second explanation relates to limitations we encountered using the proposed ACR–EULAR criteria. First of all, ANA has been proposed as entry criterium. However, in our cohort, we found nine patients with the clinical diagnosis of SLE and a sufficient amount of points for the proposed criteria, but with negative ANA. All but one of these patients also had negative anti-dsDNA. Seven of these patients had biopsy-proven lupus nephritis (LN), of which five had LN class IV, one LN class V and one LN class III. In addition, two patients had LN for which they received immunosuppressive treatment (class unknown). When ANA was not used as entry criterium, sensitivity increased to 90% (95% CI 87% to 94%), while specificity remained similar. As a consequence, using the proposed criteria would exclude patients with ANA-negative LN from (future) clinical studies. In addition, it is known from previous studies that patients with early SLE can have negative ANA as well.9 Therefore, we think that using ANA positivity as an entry criterium should be reconsidered, especially in the case of biopsy-proven LN.

Second, in the proposed ACR–EULAR criteria, the definition of the NP domain is limited. It does not specify a time correlation between NP symptoms and the (suspected) diagnosis of SLE. In our cohort, 52 of the 360 patients (one patient had two syndromes) had a positive NP domain, after NP symptoms attributed to other causes were excluded. We recalculated sensitivity and specificity after excluding patients with NP symptoms >1 year prior to (suspected) SLE, as previously proposed by Bortoluzzi et al.10 This led to the exclusion of 14 patients (26.9%). Although this did not influence the sensitivity or specificity in our cohort, we feel that accurate attribution of NP symptoms is important. This is also demonstrated in our cohort in a different way, as nine patients had definitive NPSLE (and the clinical diagnosis of SLE), but did not meet the proposed criteria and did not fulfil the NP domain. The following NPSLE syndromes were present in these patients: chorea (n=1), myelopathy (n=2), cerebral vasculitis (n=2), mood disorder (n=1), cognitive dysfunction (n=1) and cerebrovascular disease (n=3). Adding these syndromes to the NP domain would lead to a maximum increase of sensitivity to 90%, without changing specificity. If the NP domain is adjusted and ANA is not used as entry criterium in case of LN, sensitivity of the proposed ACR–EULAR criteria maximally increases to 93%.

In conclusion, we demonstrate that in our cohort of referred patients with (suspected) SLE with NP symptoms, sensitivity of the proposed ACR–EULAR criteria for SLE is high, but specificity remains suboptimal. Including patients with ANA-negative LN and specifying the NP domain might further improve the sensitivity of the proposed criteria.