Performance of the Revised Classification Criteria for Systemic Autoimmune Rheumatic Diseases and Their Overlap Syndromes.

Objective We evaluated the performance of the revised classification criteria for assessing different systemic autoimmune rheumatic diseases and their overlap syndromes. Methods A total of 652 patients with or highly suspected of having systemic lupus erythematosus (SLE), systemic sclerosis (SSc), polymyositis (PM)/dermatomyositis (DM) or rheumatoid arthritis (RA) were included in this study. The 1997 revised American College of Rheumatology (ACR) and the 2019 European League Against Rheumatism (EULAR)/ACR criteria for SLE, the 1980 ACR and the 2013 ACR/EULAR criteria for SSc, the criteria by Bohan and Peter and the 2017 EULAR/ACR criteria for PM/DM, and the 1987 revised ACR and 2011 ACR/EULAR criteria for RA were used for disease classification. Results The old and new criteria and a clinical diagnosis were used to respectively classify 103, 106 and 105 SLE patients; 35, 47 and 58 SSc patients; 18, 23 and 33 PM/DM patients; and 297, 389 and 468 RA patients. Sensitivity increased from 82.9% to 92.4% in SLE, from 56.9% to 79.3% in SSc, from 54.5% to 66.7% in PM/DM, and from 62.6% to 80.8% in RA. SLE-SSc was the predominant type of clinical overlap syndrome, while SLE-RA was the most classifiable. Conclusion The revised classification criteria for all the diseases showed an improved sensitivity, and SLE-overlap syndrome was predominant, regardless of the criteria sets.

Introduction

Systemic autoimmune rheumatic diseases (SARDs) such as systemic lupus erythematosus (SLE), systemic sclerosis (SSc), polymyositis (PM)/dermatomyositis (DM) and rheumatoid arthritis (RA) are multi-organ, inflammatory diseases that primarily affect the connective tissues and/or blood vessels. The etiology is unknown, and various manifestations as well as heterogeneous laboratory, imaging and histological findings exist among patients with the same disease.

While no diagnostic criteria for SARDs with 100% sensitivity and specificity have yet been established, even after decades of research (1,2), classification criteria that do not require a gold standard and have roughly 90% sensitivity and specificity are acceptable for the formation of patient groups for clinical studies. These criteria have undergone many revisions thus far. For example, the American College of Rheumatology (ACR) classification criteria for SLE were revised in 1997 (3), followed by the Systemic Lupus International Collaborating Clinics (SLICC) criteria reported in 2012 (4) and the European League Against Rheumatism (EULAR)/ACR criteria published in 2019 (5). The ACR preliminary classification criteria for systemic sclerosis developed in 1980 (6) and their revision, published by ACR/EULAR in 2013 (7). Bohan and Peter developed criteria for PM/DM in 1975 (8) and the EULAR/ACR criteria were reported in 2017 (9). The ACR criteria for RA were revised in 1987 (10), and the ACR/EULAR criteria were reported in 2010 (11).

When making a clinical diagnosis for individual patients, the above classification criteria are referenced. However, differential diagnoses must be considered stringently before reaching a conclusive diagnosis, due to the limited specificity of these criteria. In addition, the diagnosis of any SARD can be made even without meeting the classification criteria in a small proportion of patients because the sensitivity of the classification criteria is limited.

Thus far, more than 100 disease susceptibility genes, such as human leukocyte antigen, have been identified for RA (12). The differential expression of certain genes, such as interferon regulatory factor 5 (IRF5), signal transducer and activator of transcription 4 (STAT4), and B lymphoid tyrosine kinase (BLK), are shared between SLE and SSc (13). Patients with clinical features that meet the classification criteria for two or more SARDs are described as having “overlap syndrome” (14,15). However, while many studies validating the new classification criteria have been performed, few have simultaneously evaluated the classification criteria of several SARDs with taking into account overlap syndrome.

Therefore, in the present study, we assessed the performance of the “new” classification criteria for SLE, SSc, PM/DM and RA and compared it with that of the previous (“old”) criteria. In addition, we assessed overlap syndromes using the old and new criteria sets.

Materials and Methods

Patients

Among 963 patients with SARDs who visited Toho University Ohashi Medical Center between May 2011 and July 2017, 652 were enrolled in this study, and their medical charts were reviewed. At first presentation, they were diagnosed with - or at least highly suspected of having - one or more of the following SARDs: SLE, SSc, PM/DM or RA. The clinical diagnosis was determined by the Japan College of Rheumatology-board certified rheumatologists according to the patients' history, clinical signs and symptoms, laboratory, imaging and histopathological findings (when available and appropriate), results of full differential diagnostic procedures and reference to available classification criteria.

Study design

This was a cross-sectional and observational study. The primary endpoint was the sensitivity of the old and new classification criteria for each of the SARDs considered in this study. The study protocol was approved by the Ethics Committee of Toho University Ohashi Medical Center, and all procedures were carried out in compliance with the Declaration of Helsinki. The need for written informed patient consent was waived by the Ethics Committee of Toho University Ohashi Medical Center in view of the retrospective and observational nature of the study.

Classification criteria

For the classification of SLE, the 1997 revised ACR classification criteria (3) and the 2019 EULAR/ACR classification criteria (5) were used. Similarly, the 1980 ACR preliminary criteria (6) and the 2013 ACR/EULAR criteria (7) were used for SSc. Bohan and Peter's criteria (8) and the 2017 EULAR/ACR criteria (9) were used for PM/DM. The 1987 ACR criteria (10) and the 2010 ACR/EULAR criteria (10) were both used for RA. Overlap syndrome was defined as the fulfillment of two or more classification criteria sets among the old (3,6,8,10) or new sets (5,7,9,11).

Statistical analyses

The statistical analyses were performed using the JMP Pro software program (version 14.2; SAS Institute Japan, Tokyo, Japan). Continuous variables were presented as medians and interquartile ranges (IQRs), and binomial data were expressed as numbers and percentages.

Results

Demographic features of the enrolled patients

The median age of the 652 patients was 69 (IQR: 53-78) years old, and 531 (81.4%) of them were women (Table 1). The median disease duration was 8.4 (IQR: 4.6-15.4) years. The clinical diagnoses made by attending rheumatologists were SLE in 105 patients, SSc in 58 patients, PM in 16 patients, DM in 17 patients and RA in 468 patients.

Table 1.

Demographic and Clinical Features of the Enrolled Patients.

Baseline characteristics		Values
Female		531 (81.4)
Age, (years)		69 (53-78)
Disease duration, (years)		8.4 (4.6-15.4)
Clinical diagnosis
SLE		105 (16.1)
SSc		58 (8.9)
PM/DM		33 (5.1)
RA		468 (71.9)
Raynaud		70 (10.7)
Photosensitivity		17 (2.6)
Malar rash		51 (7.8)
Proximal scleroderma		23 (3.5)
Sclerodactyly		47 (7.2)
Heliotrope rash		7 (1.1)
Gottron’s papule		6 (0.9)
Gottron’s sign		9 (1.4)
Morning stiffness >1 hour		263 (40.3)
Symmetrical arthritis		387 (59.4)
Proximal muscle weakness		17 (2.6)
Pleuritis		32 (4.9)
Interstitial lung disease		70 (10.7)
Proteinuria		69 (10.6)
Elevated serum CK		19 (2.9)
Elevated ESR		501 (76.8)
Elevated serum CRP		469 (71.9)
ANA positive		455 (69.7)
RF positive		377 (57.8)

Values are represented as number (%) or median (IQR).

ANA: serum antinuclear antibody, CK: creatine kinase, CRP: C-reactive protein, IQR: interquartile range, PM/DM: polymyositis/dermatomyositis, RA: rheumatoid arthritis, RF: rheumatoid factor, SLE: systemic lupus erythematosus, SSc: systemic sclerosis

Raynaud's phenomenon, symmetrical arthritis, interstitial lung disease and proteinuria were observed in 70 (10.7%), 387 (59.4%), 70 (10.7%) and 69 (10.6%) patients, respectively. Serum antinuclear antibody (ANA) and rheumatoid factor (RF) were positive in 455 (69.7%) and 377 (57.8%) patients, respectively.

The comparison of old and new classification criteria for SLE, SSc, PM/DM and RA

A total of 103 and 106 patients met the old and the new SLE criteria, respectively (Figure). Similarly, 35 and 47 patients met the old and the new SSc criteria, respectively. Eleven and seven (a total of 18) patients met the old PM and DM criteria, respectively, while 12 and 11 (total of 23) patients met the new PM and DM criteria, respectively. For RA, 297 met the old criteria, whereas 389 met the new criteria. Thus, more patients were classified as per the new criteria than the old criteria.

Figure.

Venn diagrams of the patients categorized based on the clinical diagnosis and old and new classifications of systemic lupus erythematosus (SLE), systemic sclerosis (SSc), polymyositis (PM)/dermatomyositis (DM) and rheumatoid arthritis (RA). The numbers represent the patients distributed in each category. Categorization based on the clinical diagnosis is shown in light blue, the old classification is in light brown, and the new classification is in light green.

Accordingly, the sensitivity significantly increased from 82.9% [old: 95% confidence interval (CI), 77.0-87.3%] to 92.4% (new: 95% CI, 87.8-95.4%) in SLE, from 56.9% (old: 95% CI, 49.9-59.4%) to 79.3% (new: 95% CI, 73.3-80.7%) in SSc, from 54.5% (old: 95% CI, 45.9-54.5%) to 66.7% (new: 95% CI, 56.8-69.2%) in PM/DM, and from 62.6% (old: 95% CI, 61.4-63.1%) to 80.8% (new: 95% CI, 79.2-81.8%) in RA, respectively. Although this study did not intend to evaluate the specificity of old and new classification criteria, it was calculated as 97.1% (old: 95% CI, 95.9-97.9%) and 98.4% (new: 95% CI, 97.5-98.9%) in SLE, 99.7% (old: 95% CI, 99.0-99.9%) and 99.8% (new: 95% CI, 99.2-100%) in SSc, 100% (old: 95% CI, 99.5-100%) and 99.8% (new: 95% CI, 99.3-100%) in PM/DM, and 97.8% (old: 95% CI, 94.7-99.1%) and 94.0% (new: 95% CI, 90.1-96.6%) in RA.

Overlap syndrome

The clinical diagnosis of overlap syndrome was SLE-SSc in six patients (including one SLE-SSc-RA) and SLE-PM and SLE-DM in one patient each. RA-overlap syndrome with SLE, SSc, PM and DM was seen in three, two, one and two patients, respectively (Table 2). In addition, 23 and 27 patients were identified as having overlap syndrome based on the old and the new criteria sets, respectively. However, the classification criteria for RA indicated the elimination of patients with arthritis due to other diseases. The clinical diagnosis of RA-overlap was thus restricted to seropositive and erosive patients. Therefore, we also examined the number of patients with overlap syndromes other than that involving RA. The number of patients with overlap syndrome was reduced to seven (including five with SLE-SSc and two with SLE-PM) based on the old criteria sets and to six (including five with SLE-SSc and one with SLE-PM) based on the new criteria sets.

Table 2.

Distribution of Patients with Overlap Syndromes.

Diseases				Clinical diagnosis	Old classification	New classification
SLE	SSc	PM/DM	RA
+	+	+	+	0	0	0
-	+	+	+	0	0	0
+	-	+	+	0	1	1
-	-	+	+	3	2	3
+	+	-	+	1	1	1
-	+	-	+	2	0	2
+	-	-	+	3	14	16
-	-	-	+	459	279	366
+	+	+	-	0	0	0
-	+	+	-	0	0	0
+	-	+	-	2	1	0
-	-	+	-	28	14	19
+	+	-	-	5	4	4
-	+	-	-	50	30	40
+	-	-	-	94	82	84
-	-	-	-	5	224	116

Numbers in bold indicate patients with overlap syndromes diagnosed based on clinical diagnosis or classification criteria. PM/DM: polymyositis/dermatomyositis, RA: rheumatoid arthritis, SLE: systemic lupus erythematosus, SSc: systemic sclerosis

Discussion

This is the first study performed to compare the revised/new classification criteria with the previous/old version for SLE, SSc, PM/DM and RA simultaneously, thereby demonstrating an improved sensitivity. It also evaluated overlap syndrome, which should be elucidated because recent studies demonstrated the shared genetic backgrounds among SARDs (13).

Studies validating the revised criteria have been performed separately for RA (16-20), SLE (21-27) and SSc (28-30). Although most of the studies showed an improved sensitivity using the revised/new criteria, some authors have suggested a lower cut-off point to further improve the sensitivity (16,30). However, an improvement in sensitivity is inevitably accompanied by a loss of specificity, which is important for classification criteria (1,2) and must be 100% for diagnostic criteria. A simple reduction in the cut-off point may therefore not be widely acceptable.

The higher sensitivity for the diagnosis of SLE, SSc and PM/DM in the new criteria than in the older criteria is attributable to the inclusion of various manifestations and laboratory tests. Interestingly, the sensitivity of both criteria for SLE were greater than those for other diseases. This was probably because of the reference to the classification criteria before making a clinical diagnosis of SLE, due to extensive systemic and multi-organ involvement.

The clinical diagnosis as an overlap syndrome of SARDs is made in patients with manifestations and test results that can be explained by two or more types of SARDs rather than a single one. The complete fulfillment of two or more classification criteria for different SARDs is applied for clinical studies of overlap syndrome.

In addition, mixed connective tissue disease (MCTD) is an entity of SARD characterized by the presence of anti-U1-RNP antibodies and a combination of clinical manifestations satisfying the requirements for at least two of them (31-33). Five patients were clinically diagnosed with MCTD, and two of them met both the old and new criteria sets for SLE-SSc overlap. In clinical practice, the diagnosis of MCTD should be carefully considered in patients who meet the criteria for SLE, SSc and PM/DM (33).

Another concern includes the RA-overlap in SARDs. This is because the 2010 ACR/EULAR criteria exclude patients with synovitis, which can be better explained by another disease (11), while the SLE criteria exclude erosive arthritis (3). Therefore, our final investigation of overlap syndrome excluded cases of RA-overlap alone, although the clinical diagnosis of RA-overlap is made in patients with erosive arthritis and RA-specific autoantibodies, such as anti-citrullinated peptide/protein antibody.

Several limitations associated with the present study warrant mention, including the limited sample size, the lack of a precise specificity evaluation and a cohort including only Japanese patients from a single center. Nonetheless, this is the first study that simultaneously compared the revised/new classification criteria with the previous/old version for SLE, SSc, PM/DM and RA, as well as their overlap syndromes.

In conclusion, all of the revised classification criteria for SLE, SSc, PM/DM and RA showed improved sensitivity, and SLE-overlap syndrome was found to be predominant, regardless of the criteria sets. These encouraging results may support future revisions of classification criteria for the development of ideal diagnostic criteria.

The authors state that they have no Conflict of Interest (COI).

Financial Support

This work was partly supported by Health and Labour Sciences Research Grants for research on intractable diseases (The Research Team for Autoimmune Diseases) from the Ministry of Health, Labour and Welfare of Japan for H.K.