Factors Contributing to Discordance between the 2011 ACR/EULAR Criteria and Physician Clinical Judgment for the Identification of Remission in Patients with Rheumatoid Arthritis.

Remission is a primary end point of in clinical practice and trials of treatments for rheumatoid arthritis (RA). The 2011 American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) remission criteria were developed to provide a consensus definition of remission. This study aimed to assess the concordance between the new remission criteria and the physician's clinical judgment of remission and also to identify factors that affect the discordance between these two approaches. A total of 3,209 patients with RA were included from the KORean Observational Study Network for Arthritis (KORONA) database. The frequency of remission was evaluated based on each approach. The agreement between the results was estimated by Cohen's kappa (κ). Patients with remission according to the 2011 ACR/EULAR criteria (i.e. the Boolean criteria) and/or physician judgment (n = 855) were divided into three groups: concordant remission, the Boolean criteria only, and physician judgment only. Multinomial logistic regression analysis was used to identify factors responsible for the assignment of patients with remission to one of the discordant groups rather than the concordant group. The remission rates using the Boolean criteria and physician judgment were 10.5% and 19.9%, respectively. The agreement between two approaches for remission was low (κ = 0.226) and the concordant remission rate was only 5.5% (n = 177). Pain affected classification in both discordant groups, whereas fatigue was associated with remission only by physician clinical judgment. The Boolean criteria were more stringent than clinical judgment. Patient subjective symptoms such as pain and fatigue were associated with discordance between the two approaches.

INTRODUCTION

Rheumatoid arthritis (RA) is a chronic inflammatory disease associated with substantial disability after a few years of disease progression (1). The ultimate objective in management of RA is to prevent disease progression, reduce or improve disability, and maintain quality of life. This objective can often be met by early treatment with disease-modifying anti-rheumatic drugs (DMARDs) (2). Advances in the treatment of RA have enabled the target of treatment for some patients to be remission, as has been highlighted in the most recent international guidelines for RA treatment (345). In addition, the ability of a new therapy to initiate a state of remission has been increasingly appreciated; for example, this goal has become a recommended outcome for clinical trials (6).

Remission in RA has been previously defined in several ways using different measurement instruments (7). However, the stringency of these different definitions has been shown to vary widely (8). Moreover, country-wide differences in remission rates have been suggested to vary according to the measuring instruments used in the different countries (9). In 2011, the American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) developed new definitions of RA remission. These definitions were created to improve RA outcomes and decrease functional disability by ensuring more uniform reporting of outcomes (10). Previous studies that applied the 2011 ACR/EULAR remission criteria to data from observational cohorts have shown that these criteria are discordant with other classification criteria (1112). The 2011 ACR/EULAR remission criteria provide the stringent definition of remission, but may be not always concordant with the state of remission defined by physician’s clinical judgment.

The purpose of this study was to assess the extent of concordance between the 2011 ACR/EULAR criteria and physician judgment of remission and to identify factors that influence the discordance between these two approaches.

MATERIALS AND METHODS

Data source and study participants

Patients over the age of 18 who satisfied the 1987 ACR classification criteria for RA and who were due to have blood samples taken for routine evaluation were recruited consecutively by rheumatologists in 23 centers. All patients were recruited during routine clinic visits as part of the KORean Observational Study Network for Arthritis (KORONA), a database generated by rheumatologists across South Korea from July 2009. All patients provided informed consent under the institutional review board (IRB)-approved study protocol. Patients were followed up annually and completed questionnaires to establish their demographic profiles, socio-economic statuses, and disease-specific outcomes. Further details of the study have been provided elsewhere (13). A total of 3,209 patients with RA were recruited.

Prevalence of remission according to each set of criteria

The prevalence of remission was estimated based on various definitions, including disease activity score (DAS) 28 - C reactive protein (CRP) (< 2.6), DAS28 - erythrocyte sedimentation rate (ESR) (< 2.6), simplified disease activity index (SDAI) (≤ 3.3), clinical disease activity index (CDAI) (≤ 2.8), and the 2011 ACR/EULAR remission criteria, which are Boolean criteria in which patients are considered to be in remission at any time point in time if they satisfy all of the following criteria: tender joint count (TJC) ≤ 1, swollen join t count (SJC) ≤ 1, CRP ≤ 1 mg/dL, and patient global assessment (PGA) ≤ 1 (on a 0–10 scale).

Each physician was also asked to assess his/her patient disease activity levels when he/she enrolled patients in this database. After dividing the patients into remission and nonremission groups, the extent of agreement between physician judgment and each set of criteria for remission was estimated.

Factors that influence the discordance between the Boolean criteria and physician judgment for remission

Patients with remission according to the Boolean criteria and/or physician judgment (n = 855) were divided into the following three groups: concordant (remission by both Boolean criteria and physician judgment), remission based on the Boolean criteria (remission only by Boolean criteria), and remission based on physician judgment (remission only by physician judgment). After comparing the baseline characteristics of patients in each group, we performed multinomial logistic regression analysis to identify the factors that affect the discordance between the Boolean criteria and physician judgment for determining remission in patients with RA.

Statistical analysis

The prevalence of remission was evaluated based on various definitions, including the Boolean criteria, the DAS28-CRP score, the DAS28-ESR score, and physician judgment. The prevalence of remission among all the patients with RA was determined. The agreement between the different criteria was estimated by Cohen’s kappa (κ). Factors influencing the discordance between the Boolean criteria and physician judgment for remission were identified by univariate analysis using the χ2 test and ANOVA. Multinomial logistic regression analysis identified factors that influenced both discordant groups compared with the concordant group with respect to remission. All data were analyzed with SPSS 21.0 (SPSS, Chicago, IL, USA); P values < 0.05 were considered statistically significant.

Ethics statement

This study was approved by the institutional review board of Hanyang University Hospital (HYUH IRB 2009-04-003). All participants provided informed consent under a protocol approved by the institutional review board.

RESULTS

Prevalence of remission in patients with RA

The remission rate was highest when it was based on the DAS28-CRP score (47.1%). A total of 1,511 patients had remission based on the DAS28-CRP score out of all enrolled patients (n = 3,209), whereas the other 1,698 patients had nonremission. Remission rates also varied among the other sets of criteria: 28.6% of patients (n = 921) had remission based on the DAS28-ESR score, 18.0% of patients (n = 579) had remission based on the SDAI, 17.6% of patients (n = 565) had remission based on the CDAI, and 12.2% of patients (n = 393) had remission based on the Boolean criteria. The physician-judged remission rate was 19.9% (n = 639) and the nonremission rate was 80.1% (n = 2,570) (Fig. 1).

Fig. 1

Prevalence of remission in patients with RA according to each set of criteria.

DAS = disease activity score, CRP = C-reactive protein, ESR = erythrocyte sedimentation rate, SDAI = simplified disease activity index, CDAI = clinical disease activity index.

Agreement between each set of criteria and physician judgment of remission in patients with RA

The agreement rates between physician judgment and each set of criteria were in fair agreement (κ = 0.22–0.37); agreement was highest between physician judgment and the CDAI (κ = 0.37 in CDAI). When physician-judged remission was compared with Boolean criteria-judged remission, 78.9% of patients exhibited concordant classification and 21.1% of patients exhibited disconcordant classification between the two sets of criteria. Only 5.5% of all patients satisfied both sets of remission criteria. The rate of patients who satisfied the Boolean remission criteria but who were not judged in remission by physicians was 6.7%, whereas the rate of patients judged in remission by physicians but who did not satisfy the Boolean criteria was 14.4% (Table 1).

Table 1

Agreement between each set of criteria and physician judgment for RA remission

Remission criteria		DAS28-CRP		DAS28-ESR		SDAI		CDAI		Boolean criteria
		No remission	Remission	No remission	Remission	No remission	Remission	No remission	Remission	No remission	Remission
Physician judgment	No remission (%)	1,533 (47.8)	1,037 (32.3)	2,013 (62.7)	557 (17.4)	2,286 (71.2)	284 (8.9)	2,296 (71.5)	274 (8.5)	2,354 (73.4)	216 (6.7)
	Remission (%)	165 (5.1)	474 (14.8)	279 (8.7)	360 (11.2)	344 (10.7)	295 (9.2)	348 (10.8)	291 (9.1)	462 (14.4)	177 (5.5)
Kappa		0.22		0.30		0.36		0.37		0.23
P value		< 0.001		< 0.001		< 0.001		< 0.001		< 0.001

DAS = disease activity score, CRP = C-reactive protein, ESR = erythrocyte sedimentation rate, SDAI = simplified disease activity index, CDAI = clinical disease activity index.

Characteristics of patients in remission according to the Boolean remission criteria and/or physician judgment

Patients were classified into three groups according to their classification by the Boolean criteria and physician judgment. Their demographic and clinical characteristics were compared between the following three groups: remission based on the Boolean criteria (n = 216), concordant remission (n = 177), and remission based on physician judgment (n = 462). Patients who satisfied both classification approaches were younger than patients in the other groups (51.6 ± 11.6 years vs. 54.1 ± 13.2 for remission by Boolean criteria and 54.8 ± 11.4 for remission by physician judgement, P = 0.007). Moreover, the proportion of female patients was lower in the group that satisfied both criteria (73.4%) than in the group that satisfied only the Boolean criteria (78.7%) or the group that satisfied only physician judgment (83.1%) (P = 0.020). With respect to laboratory tests, rheumatoid factor (RF) positivity (78.7% vs. 75.7% and 78.1%, P = 0.748) and abnormal ESR were common (44.0% vs. 20.9% and 32.9%, respectively, P < 0.001), while the prevalence of abnormal CRP was lower (8.8% vs. 14.8% and 20.1%, respectively, P = 0.001) in the concordant group than in the two other groups (Table 2).

Table 2

Characteristics of patients in remission according to the Boolean criteria and/or physician judgment

Variables	Remission based on the Boolean criteria (n = 216)	Concordant (n = 177)	Remission based on physician judgment (n = 462)	P
Demographics
Age (years, median, IQR)	54.5, 46.0, 62.0	52.0, 44.0, 59.0	55.0, 47.0, 63.0	0.007
Female (No., %)	170 (78.7)	130 (73.4)	384 (83.1)	0.020
Disease duration (years, median, IQR)	5.1, 1.6, 10.4	5.4, 2.2, 10.1	5.7, 2.0, 11.2	0.345
Education (No., %)				0.059
Less than high school	80 (37.4)	50 (28.6)	191 (41.5)
High school	79 (34.6)	74 (42.3)	159 (34.6)
College	55 (25.7)	51 (29.1)	110 (23.9)
Income level (No., %)				0.304
≤ $19,999	94 (43.7)	64 (36.4)	207 (44.8)
$20,000–$49,999	87 (40.5)	76 (43.2)	170 (36.8)
≥ $50,000	34 (15.8)	36 (20.5)	85 (18.4)
BMI (No., %)				0.297
Underweight, < 18.5	24 (11.2)	11 (6.2)	30 (6.5)
Normal, 18.5–22.9	104 (48.4)	83 (46.9)	241 (52.3)
Pre-obese, 23–24.9	48 (22.3)	44 (24.9)	97 (21.0)
Obese, ≥ 25	39 (18.1)	39 (22.0)	93 (20.2)
Laboratory (No., %)
Rheumatoid factor positive	170 (78.7)	134 (75.7)	361 (78.1)	0.748
CRP positive	32 (14.8)	15 (8.5)	93 (20.1)	0.001
ESR positive	95 (44.0)	37 (20.9)	152 (32.9)	< 0.001
Medication (No., %)
Methotrexate	186 (86.1)	151 (85.3)	364 (78.8)	0.039
Biologic DMARD	9 (4.2)	12 (6.8)	42 (9.1)	0.069
Glucocorticoid	138 (63.9)	95 (53.7)	272 (58.9)	0.122
NSAID	107 (49.5)	69 (39.0)	206 (44.6)	0.112
Patient outcome (Mean ± SD)
Tender joint count	0.27 ± 0.44	0.12 ± 0.32	1.62 ± 3.13	< 0.001
Swollen joint count	0.19 ± 0.39	0.06 ± 0.24	0.66 ± 1.75	< 0.001
HAQ-DI	0.17 ± 0.28	0.09 ± 0.21	0.38 ± 0.45	< 0.001
Fatigue (10 cm VAS)	1.6 ± 2.04	1.42 ± 1.98	3.91 ± 2.86	< 0.001
Sleep disturbance (10 cm VAS)	0.62 ± 1.39	0.50 ± 1.18	1.91 ± 2.53	< 0.010
Pain (10 cm VAS)	0.75 ± 1.18	0.47 ± 0.70	2.55 ± 2.33	< 0.001

Concordant: remission by both Boolean criteria and physician judgment, Remission based on the Boolean criteria: remission only by the Boolean criteria, Remission based on physician judgment: remission only by physician judgment.

SD = standard deviation, BMI = body mass index, CRP = C-reactive protein, ESR = erythrocyte sedimentation rate, DMRAD = disease modifying anti-rheumatic drug, NSAID = non-steroidal anti-inflammatory disease, HAQ-DI = health assessment questionnaire disability index, VAS = visual analogue scale.

Factors that influence discordance between classification by the Boolean criteria and physician judgment for remission in patients with RA

Next, multinomial logistic regression analysis was performed to identify factors that influence the extent of discordance between classification by the Boolean criteria and physician judgment for remission, with the concordant group as reference. This analysis used variables found to be significant in univariate analysis (P < 0.05), such as age, sex, methotrexate use, RF positivity, abnormality of ESR or CRP, degree of fatigue, degree of sleep disturbance, and degree of pain.

The factors associated with remission only by the Boolean criteria were: abnormal ESR (odds ratio [OR], 2.72; 95% confidence interval [CI], 1.69–4.38) and a high level of pain (OR, 1.38; 95% CI, 1.02–1.85). The factors associated with remission only by physician judgment were: old age (OR, 1.03; 95% CI, 1.01–1.05), abnormal CRP (OR, 3.13; 95% CI, 1.50–6.50), fatigue (OR, 1.24; 95% CI, 1.12–1.38), and a high level of pain (OR, 3.13; 95% CI, 2.30–4.27) (Table 3).

Table 3

Factors that influence discordance between the Boolean criteria and physician judgment for remission*

Variables	Remission based on the Boolean criteria				Remission based on physician judgment
	Coefficient	SE	P	Odds ratio (95% CI)	Coefficient	SE	P	Odds ratio (95% CI)
Age	0.02	0.01	0.088	1.02 (0.99–1.03)	0.03	0.01	0.012	1.03 (1.01–1.05)
Female	0.46	0.27	0.085	1.58 (0.94–2.67)	0.35	0.28	0.212	1.41 (0.82–2.43)
Methotrexate use	0.10	0.31	0.755	1.10 (0.60–2.01)	-0.65	0.30	0.031	0.52 (0.29–0.94)
Negative RF	-0.08	0.26	0.759	0.92 (0.56–1.53)	0.32	0.26	0.223	1.37 (0.83–2.29)
Abnormal ESR	1.00	0.24	< 0.001	2.72 (1.69–4.38)	0.07	0.28	0.817	1.07 (0.61–1.86)
Abnormal CRP	0.40	0.36	0.256	1.50 (0.75–3.00)	1.14	0.37	0.002	3.13 (1.50–6.50)
Fatigue	-0.01	0.06	0.857	1.01 (0.90–1.14)	0.22	0.05	< 0.001	1.24 (1.12–1.38)
Sleep disturbance	0.05	0.09	0.615	1.05 (0.87–1.26)	0.17	0.08	0.037	1.18 (1.01–1.38)
Pain	0.32	0.15	0.036	1.38 (1.02-–1.85)	1.14	0.16	< 0.001	3.13 (2.30–4.27)

RF = rheumatoid factor, ESR = erythrocyte sedimentation rate, CRP = C-reactive protein.

*The total of 855 includes the concordant group (177 patients) as the reference group.

DISCUSSION

In the present study we found that 12.2% of all patients with RA in routine clinical care achieved remission by the 2011 ACR/EULAR remission criteria (i.e. the Boolean criteria). Compared with the other sets of remission criteria examined in this study (DAS28-ESR, DAS28-CRP, CDAI, SDAI, and physician clinical judgment), the Boolean criteria were the most stringent for identifying patients with RA remission. Poor agreement was observed between all sets of composite criteria and physician judgment for remission; the Boolean criteria showed the lowest level of agreement with physician judgment for remission. Patient subjective symptoms such as pain and fatigue, in addition to laboratory results such as CRP and ESR, were associated with discordance between classification by the Boolean criteria and physician judgment for remission.

In clinical practice, it is crucial to reach a consensus definition for remission in patients with RA, since the current treatment goal for RA is to achieve and maintain this state (345). Recently, treat-to-target strategies for RA treatment have been widely accepted, and periodic monitoring of disease activity of RA patients has been emphasized (345). However, despite the compelling evidence supporting this approach, adherence to the treat-to-target strategy in clinical practice is often low. This non-adherence is thought to arise from differences between the various composite measures of disease activity and physician judgment of remission. In the treat-to-target strategy, DAS, DAS28, CDAI, or SDAI should be employed to assess the disease activity of each patient. However, here we showed that the extent of agreement with each of these methods with physician judgment for remission varies widely. The agreement between CDAI and physician judgment was the highest. We hypothesize that this high level of agreement is because the CDAI includes a physician global assessment (0–10 cm) scale for estimating disease activity. In contrast, DAS28-CRP, the Boolean criteria, and physician judgement showed the lowest levels of agreement. This finding can be explained by a different reason; that is, the DAS28-CRP criteria are relatively loose, while the Boolean criteria for remission are more stringent.

Although either set of criteria can be used to define remission in patients with RA, the Boolean criteria may be most useful for predicting maintenance of remission or the possibility of drug discontinuation for patients who are in sustained remission, due to their stringency (1415). Hence, it is important to understand the factors that affect discordance between the Boolean criteria and physician judgment of remission to decrease the gap between them.

We found that elevated ESR and high levels of subjective pain reported by patients were both associated with remission in patients who satisfied only the Boolean criteria, not physician judgment. Since the Boolean criteria included only CRP (and not ESR) as an inflammatory marker, this result is reasonable. Moreover, subjective patient pain does not always mean high disease activity (16). Therefore, if the pain felt by patients is communicated to physicians, pain can be a factor that influences discordance between the Boolean criteria and physician judgment.

On the other hand, the factors associated with patients in remission by physician judgment but not by the Boolean criteria were increased CRP, old age, and increased pain. We note that the Boolean criteria use a CRP cutoff of 1 mg/dL; however, our normal level in usual practice ranges from 0.3 mg/dL to 0.8 mg/dL at the different hospitals. When the measured CRP level is over the cut-off level at a particular hospital, physicians usually do not assert that such a patient is in remission. The data regarding old age and subjective pain point to another interesting observation, namely, older patients do not easily talk about their disease activity. We speculate that this barrier might influence patient-physician communication. Moreover, if physicians do not ask cautiously about patient feelings or global health, they may tend to judge disease activity of their patients based on objective physician examinations or lab test results. Previous reports have also shown that age and comorbidity can influence self-reported disability or pain; hence, these factors also need to be taken into account when interpreting remission or response rates in RA (17). Pain was associated with both discordances. However, its impact was stronger in the physician judgment classification as opposed to the Boolean criteria (OR, 3.13) than it was in remission in patients who satisfied only the Boolean criteria but not physician judgment (OR, 1.38). This finding indicates that pain has a greater influence on patient perception than on physician judgment. Previous reports have shown that patient-reported global health is influenced by pain, fatigue, and comorbid conditions (181920). These factors may cause some patients to persistently score high on their global assessment components, even though they have low numbers of active joints and normal acute phase reactants. Ultimately, this discrepancy can lead to discordance about remission classification between methods that include patient-reported outcomes vs. physician judgment. Therefore, improving physician-patient communication about disease activity and pain holds promise to resolve this discordance between the Boolean criteria and physician judgment for classifying patients with RA in remission. Through this effort, patient-reported outcomes will be reflected in the decision-making by their physicians.

There are several limitations to our study. First, there is no official consensus about remission by physician judgment. Each physician has different background and experience; thus, physician judgment for remission may vary from person to person. However, it made physicians to define patients in remission advertently. The relatively low proportion of patients in remission by physician judgment can be explained by their stringent tendencies. Therefore, we considered it appropriate to compare two conservative definitions of remission (Boolean criteria vs. physician judgment). Second, in practice, physician judgment is usually based on both the current and past statuses of patients. Thus, it is inevitable that some differences will arise between physician judgment and the Boolean criteria. However, from this perspective, the Boolean criteria are considered to be superior to the other composite indices for identifying true patients in remission, because these criteria do not add individual values or constants to calculate disease activity. Because patient global assessments can reflect both the current and past statuses of disease activity of each patient, we expect that further follow-up studies would allow us to determine the extent of agreement between changes in disease activity and maintenance of remission. Third, we did not analyze the extent of discordance for each disease activity level. Since this study was conducted to compare each set of remission criteria (including the Boolean criteria) with physician judgment, we did not estimate the extent of agreement of each composite index to classify each level of disease activity. Since low disease activity is another aim for RA treatment, further analysis aimed at identifying patients with low disease activity will be helpful for informing decision making in clinical practice.

In conclusion, here we found that the Boolean criteria are more stringent than physician clinical judgment. Moreover, these two approaches have a low extent of agreement. Patient subjective symptoms such as pain and fatigue were associated with discordance between the two approaches.