Utility of the new rheumatoid arthritis 2010 ACR/EULAR classification criteria in routine clinical care.

OBJECTIVES: The new 2010 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) criteria for rheumatoid arthritis (RA) have been designed to classify early onset RA, but has not been studied to identify RA in patients with arthritis seen in routine clinical care where correct 'classification' of patients, when they are not selected for having RA would be important.
DESIGN: Prospective, consecutive patients cohort.
SETTING: Outpatient clinic of a university rheumatology centre. PARTICIPANTS: A total of 126 patients with joint symptoms were consecutively recruited.
INTERVENTIONS: The ACR/EULAR RA criteria were applied, with questions followed by a targeted musculoskeletal exam. The gold standard for the diagnosis of RA was the primary rheumatologist's diagnosis. PRIMARY OUTCOME MEASURE: Number of patients with non-RA diagnosis who were classified as having RA by the new classification criteria.
RESULTS: The sensitivity and specificity of the 2010 criteria in classifying RA were 97% and 55%, respectively, compared with the 1987 RA criteria which were 93% and 76%, respectively. The 2010 criteria as applied to this group of patients had a poorer positive predictive (44% vs 61%) and a similar negative predictive value (98% vs 97%) compared with the 1987 criteria. More specifically, 66.7% of systemic lupus erythematosus patients, 50% of osteoarthritis, 37.5% of psoriatic arthritis and 27.2% of others fulfilled the new criteria and could have been classified as RA.
CONCLUSIONS: In this, we believe, the first study to examine the new 2010 ACR/EULAR RA criteria among consecutive patients seen in routine care, we found the criteria to have low specificity, and therefore incorrectly label those as having RA when, in fact, they may have a different type of inflammatory arthritis. Physicians need to be aware of this when applying the new criteria for classifying their patients for any purpose.

The new rheumatoid arthritis (RA) classification criteria have not been tested in disease controls yet.

Without determining whether classification criteria differentiate among patients with similar symptoms and correctly classify them, it is not possible to have a clear understanding of how they will perform.

The new RA criteria classify about 50% of patients with a non-RA diagnoses as having RA.

Specificity of the new criteria is much worse than the old criteria and may lead to over and misdiagnosis of patients with RA, leading to inappropriate medication use.

The strength of our study is that we included all comers to a busy rheumatology clinic and used the doctor's diagnosis as the gold standard, different from using medication start, as was done in the new criteria, which is one of the main problems of the criteria.

Our main limitation is that we studied 112 patients and our results could have been more robust with a larger number of patients. Also not all patients were early RA patients.

Introduction

Rheumatoid arthritis (RA) affects 1% of the population and leads to the progressive disability and joint destruction if not diagnosed and treated early.1 Early treatment impacts long-term clinical improvements2 3 and early and aggressive treatment of RA has revolutionised our approach to how we use our current medications. Ongoing trials are constantly searching for novel therapeutic targets to treat this debilitating disease with the aim to further improve the long-term outcome for our RA patients. Achieving remission seems most likely, if the inflammatory burden is turned off as soon as after symptoms start; also the amount of time a patient spends in remission is likely determined by how early this response was achieved. Therefore, it is necessary to have the ability to classify patients early in their RA course in order to investigate new and appropriate treatments and treat them early in every day clinical care.

Previously, the American College of Rheumatology (ACR) 1987 RA criteria4 have been used to classify patients as RA for participating in studies and it also extended into clinical practice for diagnostic purposes.5 However, these criteria have been shown to have a low sensitivity for targeting early arthritis and will therefore miss the crucial population in which disease progression can be halted.4 Especially, rheumatoid nodules and radiographic findings are rare in early disease, in addition to less symmetric disease early on in the course of RA.

The new 2010 ACR/EULAR (European League Against Rheumatism) criteria for RA have been designed to classify patients with early RA in order to better study this group of patients.6–8 Patients who would benefit from rapid initiation of methotrexate therapy are targeted using the new criteria. The new RA classification criteria state that, as a prerequisite, a patient needs to have a swollen joint and ‘…all other possible diagnosis have been eliminated’.6 This is an interesting statement as, if all other possible causes were so simple to identify and eliminate, we would not need criteria for RA as that would be the only diagnostic option left to the clinician. Many inflammatory conditions have overlapping and similar symptoms, and classification criteria need to demonstrate that they can differentiate among these conditions to be really useful in identifying the ‘true’ RA patients.

It is, however, not yet known how these criteria for the classification of RA perform to identify RA in patients with arthritis seen in routine clinical care as well as in patients with other conditions that may mimic RA. This was not part of the criteria development process, a major shortcoming in our opinion. Illnesses such as systemic lupus erythematosus (SLE), psoriatic arthritis (PsA) and osteoarthritis (OA), can present with inflammatory joints and positive laboratory testing results that may imitate RA on a scoring system. It therefore becomes necessary to test a diagnostic tool in a broad spectrum of patients, including a control population of non-RA rheumatic conditions, in order to be appropriately validated.9 Likewise, many patients who present with early undifferentiated arthritis may go on to develop into other diseases and it must be determined whether the criteria have the ability to distinguish among them. The 2010 classification criteria may also similarly be adapted into a diagnostic criteria by practising physicians as the 1987 criteria have been and the appropriateness of its use should be addressed.

The objective of this study, therefore, is to test the validity of the new 2010 ACR/EULAR RA criteria prospectively in a general rheumatology practice to determine how well it can distinguish RA from other types of inflammatory arthritis.

Methods

Patients were evaluated during their usual care in a general rheumatology private practice of seven rheumatologists at a university setting. The investigators screened patients one day a week, at the New York University Hospital for Joint Diseases between 1 August 2010 and 30 April  2011. A total of 126 new and established consecutive patients were recruited after being screened by their primary rheumatologist. Patients were included if they had any joint symptoms between the stated dates, regardless of prior patient diagnosis, and if they consented to the study. No patients were excluded from the screening protocol. The study was approved by local Institutional Review Board for the Protection of Human Subjects.

The 2010 ACR/EULAR RA criteria were applied, as laid out by the task force, by a single rheumatologist at the patient's initial visit. Questions pertaining to the 1987 and 2010 RA classification criteria were asked to the participants, as well as classification criteria for SLE (ACR), PsA (CASPAR) and OA of the knee (ACR clinical, radiographic or laboratory). This was followed by a targeted exam of the musculoskeletal system, including joint counts, as well as an exam for extra-articular manifestations. The patients were followed retrospectively and prospectively by chart review for additional examination findings, laboratory data (rheumatoid factor (RF), anticyclic citruilinated peptide antibody (ACPA), erythrocyte sedimentation rate (ESR), C reactive protein (CRP), antinuclear antibodies, antidouble-stranded DNA antibodies, antismith antibodies, antiphospholipid antibodies, complete blood count and urinalysis), radiographic tests (hand and foot radiographs) and physician diagnosis not present at time of screening.

Statistical analysis

Summary statistics were estimated for demographic and clinical feature data by 2010 ACR/EULAR RA criteria; differences between groups were evaluated using the Wilcoxon Rank Sum test for continuous data and Fisher's exact test for discrete/categorical data. The predictive acumen of the 2010 ACR/EULAR RA criteria was determined by comparing its classification against the ‘gold standard’, the experienced primary rheumatologist's diagnosis as recorded in the patient's chart. 1987 ACR/EULAR RA criteria classification was also determined for comparison. Receiver–operator characteristic (ROC) curves and area-under-the-curves (AUCs) were estimated after logistic regressions in which either 1987 or 2010 ACR/EULAR RA criteria predicted physician diagnosis. Estimates of agreement existing beyond that of chance alone between 1987 and 2010 ACR/EULAR RA criteria and physician diagnosis was estimated using the Kappa statistic. Analysis was completed using SAS V 9.3 (Cary, NC, USA) and Stata V 12.0 (College Station, TX, USA).

Results

A total of 126 patients were recruited for the study. Fourteen patients were excluded from analysis due to inadequate data regarding laboratory testing needed to calculate the 2010 RA criteria (no RF, ACPA, ESR and/or CRP; figure 1) Patients who did not have complete laboratory data but still met the criteria based on a score of 6 or more were included. Therefore, 112 participants were included in the report. They had a mean age of 48.2 years (SD=15), with 77.7% (n=87) women, an average duration of disease of 5.3 years (SD=6.3) and 25% (n=28) were new patients diagnosed within the last 6 months. Patients were followed for an average of 6.3 months (SD=2.6). Twenty-eight patients (25.0%) were on non-steroidal anti-inflammatory drugs (NSAIDS) during the study, the majority of patients (n=96, 85.7%) were on additional medications, with 51 (45.5%) on corticosteroids, 82 (73.2%) on disease-modifying antirheumatic drugs (DMARD), 41 (36.6%) on biological and 6 (5.4%) on cytotoxic agents (table 1).

Figure 1

Flow diagram of subjects.

Table 1

Demographic and clinical features of 112 patients by 2010 ACR/EULAR RA criteria classification

	2010 ACR/EULAR RA criteria
	Classified as RA	Not classified as RA	p Value
N	66	46
Age (years)	49.5 (15.5)	46.3 (14.1)	0.318
Gender (N (%))			0.492
Female	53 (80.3%)	34 (73.9%)
Male	13 (19.7%)	12 (26.1%)
Disease duration (years)	6.0 (6.0)	4.3 (6.5)	0.007
Follow-up (months)	6.1 (2.7)	6.6 (2.4)	0.347
New patient (N (%))	11 (16.7%)	17 (37.0%)	0.025
Physician diagnosis (N (%))			<0.001
(RA)	29 (43.9%)	1 (2.2%)
Systemic lupus erythematosus	16 (24.2%)	8 (17.4%)
Psoriatic arthritis	9 (13.6%)	15 (32.6%)
Osteoarthritis	6 (9.1%)	6 (13.0%)
Other	6 (9.1%)	16 (34.8%)
Medication usage (N (%))
NSAID	17 (25.8%)	11 (23.9%)	0.999
Steroid	35 (53.0%)	16 (34.8%)	0.082
DMARD	55 (83.3%)	27 (58.7%)	0.005
Biological	30 (45.5%)	11 (23.9%)	0.028
Cytotoxic	4 (6.1%)	2 (4.4%)	0.999
Rheumatoid factor (RF) (N (%))			<0.001
Positive	30 (45.6%)	3 (6.5%)
Negative	36 (54.4%)	43 (93.5%)
RF/ACPA (N (%))			<0.001
Negative	29 (47.5%)	38 (90.5%)
Low titre	7 (11.5%)	4 (9.5%)
High titre	25 (41.0%)	0 (0.0%)
CRP/ESR (N (%))			0.014
Positive	51 (77.3%)	25 (54.3%)
Negative	15 (22.7%)	21 (45.7%)
Joint count (N (%))			<0.001
1–3 Small joints	2 (3.0%)	10 (21.7%)
4–10 Small joints	10 (15.2%)	16 (34.9%)
10+ Small joints	54 (81.8%)	0 (0.0%)
1 Large joint	0 (0.0%)	10 (21.7%)
2–10 Large joints	0 (0.0%)	10 (21.7%)

CRP, C reactive protein; DMARD, disease modifying antirheumatic drugs; ESR, erythrocyte sedimentation rate; NSAID, non-steroidal anti-inflammatory drug; RA, rheumatoid arthritis.

Thirty (26.8%) patients had a primary diagnosis of RA made by their rheumatologist. The remainder had diagnoses of SLE (n=24, 21.4%), PsA (n=24, 21.4%), OA (n=12, 10.7%) and other rheumatic conditions (n=22, 19.6%)—spondyloarthritis (SpA) (n=8, 7.1%), undifferentiated autoimmune syndrome (UAS) (n=4, 3.6%), Sjogren's syndrome (SS) (n=3, 2.7%), gout (n=2, 1.8%), polymyalgia rheumatic (PMR) (n=1, 0.9%), undifferentiated inflammatory arthritis (n=1, 0.9%), mixed connective tissue disease (MCTD) (n=1, 0.9%), vasculitis (n=1, 0.9%) and Behcet's syndrome (n=1, 0.9%).

When the 2010 criteria were applied blindly to the patients, despite manifestations of other diseases that may have ruled out a diagnosis of RA, 66 (58.9%) did meet the ACR/EULAR 2010 RA criteria (score >=6) and 46 (41.1%) did not. In those with a known diagnosis of RA, 29 (96.7%) met the 2010 criteria. In the combined population of those without a rheumatologist diagnosis of RA (n=82), 37 (45.1%) fulfilled the new criteria. Among patients with other rheumatic diseases, 16 (66.7%) SLE patients, 8 (37.5%) PsA, 6 (50%) OA and 6 (27.2%) additional patients (two SS, two SpA, one gout and one MCTD) fulfilled the new criteria. Of the 28 subjects who newly presented to the office (within less than 6 months of study participation), 11 (39.3%) fulfilled the new criteria (2 with RA and 9 with other diseases—3 SLE, 3 PsA and 3 OA) and 17 did not.

The features of patients who fulfilled the 2010 criteria versus those who did not are shown in table 2. Most of the patients who did not fulfil the criteria had a negative rheumatoid factor or ACPA and a joint count including larger joints or few small joints. However, of those that did fulfil the criteria, only about 49% were positive for high or low titre RF or ACPA while 82% had more than 10 small joints with synovitis or tenderness.

Table 2

Comparison of sensitivity and specificity of 2010 vs 1987 rheumatoid arthritis criteria

	2010 Criteria	1987 Criteria
Sensitivity (%)	97	93
Specificity (%)	55	76
Positive predictive value (%)	44	61
Negative predictive value (%)	98	97

We examined the same patients with application of the 1987 criteria in order to compare the two classification systems. Twenty-eight (93.3%) patients with RA fulfilled the 1987 criteria, as well as 18 (24.3%) without RA, 7 (38.9) with SLE, 7 (30.4) with PsA, none with OA and 4 (18.2%) with additional diagnoses.

The sensitivity and specificity of the 2010 criteria in classifying RA were 97% and 55%, respectively, compared with the 1987 RA criteria which were 93% and 76%, respectively. The 2010 criteria as applied to this group of patients had a poorer positive predictive (44% vs 61%) and a similar negative predictive value (98% vs 97%) versus the 1987 criteria (table 2).

The ROC was calculated (figure 2). The sensitivity and specificity of the criteria in new patients, with less than 6 months of symptoms or two or less visits, were 100% and 65%, respectively. However, the number of new patients was limited, which restricts the use of this analysis.

Figure 2

Receiver–operator characteristic curves of ACR/EULAR RA criteria predicting MD diagnosis of ‘rheumatoid arthritis’.

We also applied criteria for SLE, PsA and OA to the subjects to test the accuracy of the diagnostic methods. Of those subjects who were diagnosed by their physician with SLE, 22 fulfilled the ACR SLE criteria along with four who did not have SLE and two with SLE did not fulfil the criteria, leading to a sensitivity and specificity of 92% and 95%, respectively. For those with PsA, 23 fulfilled the CASPAR criteria, one did not, and one person without PsA fulfilled the criteria, giving a sensitivity of 96% and specificity of 99%. For OA of the knee, 27 patients had a primary or secondary diagnosis of OA and 24 fulfilled the ACR knee OA criteria (either clinical, radiographic or laboratory), along with 29 who did not carry a diagnosis of OA. This had a sensitivity of 92% and specificity of 65%. These are all similar to previous publications for SLE, PsA and OA.10–12

Discussion

Diagnosis can be seen as the classification of an individual patient and the distinction between ‘classification’ and ‘diagnostic’ criteria is somewhat artificial.13 14 Therefore, it is of interest to know how any classification criteria will perform in the real world where diagnosis and classification go hand in hand.

Our aim was to test the real-world performance of the new criteria in a mixed rheumatology patient population. We believe this is one of the first studies to examine the new 2010 ACR/EULAR criteria in a broad spectrum of rheumatological diseases among consecutive patients seen in routine rheumatology care.

We report a sensitivity and specificity of 97% and 55%, respectively, for the 2010 criteria and a sensitivity and specificity of the 1987 criteria of 93% and 76%, respectively, which are similar to previous reports.4 15 16 The 2010 criteria are more sensitive in classifying RA patients compared with the 1987 criteria. However, the criteria have a low specificity that may not distinguish between the different types of inflammatory arthritis. Almost half of patients (45.1%) without a rheumatologist's diagnosis of RA meet criteria using the 2010 classification.

The 2010 criteria were developed in order to improve upon the shortcomings of the 1987 criteria which overlooked an important new RA population who would benefit from early therapeutic intervention. Several previous studies have compared the 2010 criteria to the 1987 criteria, and that by Kaneko et al17 reported a sensitivity of 47.1% for the 1987 compared with 73.5% for the 2010. The Leiden early arthritis group calculated a sensitivity of 74% in an undifferentiated arthritis cohort with a gold standard of DMARD initiation or a sensitivity of 84% with methotrexate therapy as the outcome.18 Also in an early arthritis group, the 2010 criteria identified an increased number of patients needing DMARDs than the 1987 criteria.15 These studies link the 2010 criteria with an improved sensitivity to better target early onset RA.

Although there is a benefit of improved sensitivity, a loss of specificity is the result frequently. Specificity in the above early RA population dropped to 71.4% for 2010 compared with 92.9% using the 1987 criteria.17 The Leiden group found between 60% and 74% for specificity using the 2010 measurement. (18) Therefore more subjects may be labelled as RA initially and on follow-up diagnosed with an alternate disorder. For example, using the 2010 compared with 1987 criteria, three times as many patients were labelled as RA; however, their disease regressed without ever commencing DMARD therapy.19

The 2010 classification criteria may very well have a positive effect for rheumatologists. The higher sensitivity may sway more primary care physicians to question a diagnosis of RA and may lead to more rheumatology referrals. This would allow rheumatologists to classify RA earlier, enrol in trials designed for undifferentiated arthritis and start treatment sooner. With a more rapid initiation of therapy, we may see even better long-term outcomes.

It must be noted, however, that with this decreased specificity and poorer positive predictive value will have important consequences. This would imply that the 2010 criteria will be less able to distinguish RA from other types of inflammatory arthritis and will incorrectly label patients with other types of arthritis as having RA. While a patient may benefit from early DMARD therapy no matter what type of inflammatory arthritis they may have, their true underlying disease might then be masked which could lead to harmful consequences of an unknown course of illness, long-term prognosis, comorbidities or incorrect usage of biologics. In one study, 50% of patients fulfilling criteria, and therefore being started on DMARDs, were eventually labelled as non-RA.16

The limitations of the current study stem from the fact that less than a quarter of the patients were newly established with early undifferentiated arthritis who are the key targets for which the criteria were developed. This was due to enrolment availability at the outpatient centre where the study was performed. However, including more established patients allowed us to better examine a broad spectrum of differentiated arthritis that may mimic RA, whereas other studies may have excluded the same patients. Furthermore, the criteria state that other diseases should be ruled out before applying the test. We purposely did not rule out other conditions in order to study our objective of test performance in a range of rheumatic illnesses. If a physician must first rule out confounding diseases before applying the criteria, this would lead to circular reasoning—whereby, if it was known that a patient did or did not have another condition, then the RA criteria would not be needed. We also used the gold standard of RA diagnosis as the expert rheumatologist's diagnosis. This is a potential limitation in methodology in any trial looking at RA diagnosis, since there is no definitive gold standard test available with current technology.

The 2010 ACR/EULAR RA criteria have been found to have a low specificity in this and several other studies. It is therefore presumed that it will incorrectly label those as having RA when in fact they may have another type of inflammatory arthritis. The importance of a thorough evaluation for confounding illnesses is stressed by these results and further evaluation of the new criteria need to include disease controls to better reflect the utility of the criteria. Physicians need to be aware of this when applying the new criteria for classifying their patients for any purpose.