Evaluation of the Disease Activity Score in Twenty-Eight Joints-Based Flare Definitions in Rheumatoid Arthritis: Data From a Three-Year Clinical Trial.

OBJECTIVE: To assess the flare rate using published criteria (Disease Activity Score in 28 joints [DAS28-2] increase between visits of >1.2 or >0.6 if current DAS28 ≥3.2) in patients receiving constant treatment, and to compare published flare criteria to criteria used by study investigators after biologic treatment discontinuation in the ACT-RAY study.
METHODS: Patients with rheumatoid arthritis (n = 553) were randomized to add tocilizumab to ongoing methotrexate, or switch to tocilizumab plus placebo. If DAS28 ≤3.2 occurred at week 24, treatment remained constant until week 52; here we assessed the DAS28-2 flare rate. Between weeks 52 and 104, patients in sustained remission (DAS28 <2.6 at 2 consecutive visits 12 weeks apart) discontinued tocilizumab and were assessed every 4 weeks. Per protocol, flare was defined as a worsening of disease activity that required treatment beyond the permitted therapy based on investigator opinions (investigator flare) and was compared with the DAS28-2 definition.
RESULTS: After tocilizumab discontinuation, DAS28-2 was sensitive (88-100%), but not specific (57-65%), for detecting investigator flare. Under constant treatment, DAS28-2 criteria were met in 136 cases per 100 patient-years despite stable disease activity. Sustained flares were infrequent. Other DAS28-based criteria led to similar conclusions.
CONCLUSION: DAS28-based flare occurred more often than investigator-defined flares after biologic agent discontinuation. More stringent criteria may be more appropriate for clinical practice.

RCT Entities:   Population Interventions Outcomes

Introduction

In rheumatoid arthritis (RA), the development of disease flare criteria is an ongoing effort, serving relevant purposes such as evaluating the duration of response after medication withdrawal. In addition, the flare rate could be a standard outcome measure for clinical research assessing stable treatment conditions. In this context, Outcome Measures in Rheumatology (OMERACT) has launched an initiative aimed at defining flare 1. A preliminary investigation of 6 proposed Disease Activity Score in 28 joints (DAS28)–based criteria identified an increase in DAS28 >1.2 or >0.6, if DAS28 ≥3.2, as the most discriminating and valid criterion 2.

Objective disease flare criteria for rheumatoid arthritis requires an understanding of how rheumatologists view the occurrence of flares and the frequency of patients meeting flare criteria based on disease activity when under constant treatment. Using data from a large randomized study of treat‐to‐target strategies based on tocilizumab (TCZ) and methotrexate, this analysis 1) compared published disease activity–based flare criteria (Disease Activity Score in 28 joints [DAS28]‐2: increase in DAS28 between visits of >1.2 or >0.6 if current DAS28 ≥3.2) to criteria used by study investigators after discontinuation of TCZ, and 2) assessed the published criteria in a closely monitored group of patients receiving constant TCZ‐based treatment according to the study protocol.

DAS28‐defined flares occurred more often than flares identified by investigators after biologic agent discontinuation. More stringent criteria may be more appropriate for clinical practice.

Under constant treatment, despite stable disease activity and minimal efficacy related withdrawals, DAS28‐2–defined criteria were met quite often; however, criteria were met mostly in relation to disease activity fluctuations isolated to a single visit. Clinical intervention may be more appropriate after flare is observed at consecutive visits.

Two specific phases of the ACT‐RAY study offer a unique opportunity to investigate relevant aspects of flares: 1) assessment of the flare rate (according to published criteria [2]) between weeks 24 and 52, during which a large patient subgroup received constant treatment 3, 4, and 2) comparison (using sensitivity and specificity) of the flare criteria implicitly used by the investigators with published criteria 2 in the second year, during which patients reaching sustained DAS28 remission discontinued tocilizumab (TCZ) and subsequently other treatments, which were re‐started after investigators diagnosed a disease flare based on their clinical judgment.

Patients and methods

Study design and procedures

ACT‐RAY was a 3‐year, double‐blind, placebo‐controlled, parallel‐group clinical trial (NCT00810199) 3, 4. Briefly, eligible patients had active RA with DAS28 using the erythrocyte sedimentation rate (ESR) of >4.4 at baseline, and received methotrexate (MTX) for ≥12 weeks, with a stable dosage of ≥15 mg/week. All patients received open‐label TCZ 8 mg/kg intravenously every 4 weeks, and were randomized to double‐blind treatment with MTX or placebo at a dose corresponding to the pre‐study period. Dose modifications of TCZ and MTX were allowed only for safety reasons or in the context of the treatment step‐down (see below). Stable dosages of oral corticosteroids (≤10 mg/day prednisone equivalents) and nonsteroidal antiinflammatory drugs were permitted.

After week 24, for patients who did not achieve low disease activity (LDA; DAS28‐ESR >3.2), a treat‐to‐target approach was used, including the addition of open‐label disease‐modifying antirheumatic drugs (DMARDs). For patients in LDA, treatment continued unchanged through week 52, except for an optional steroid reduction not below 5 mg/day of prednisone equivalents.

After week 52, the strategy included a treatment step‐down approach: patients who achieved sustained clinical remission (DAS28‐ESR <2.6 at 2 consecutive visits 12 weeks apart) progressively discontinued study treatments: first TCZ and subsequently, if still in sustained remission, the open‐label DMARD (where present) and the blinded medication (MTX/placebo).

After treatment discontinuation, patients were monitored for flares for 1 year. Flares were not defined by a particular disease activity level; a flare was considered a worsening of the subject's disease activity such that, in the investigator's opinion, it required treatment intensification going beyond the permitted supportive therapy (termed “investigator flare”). Flares could occur at or between visits that were scheduled every 4 weeks, with assessments including the components of DAS28 and the American College of Rheumatology core data set.

Statistical analysis

In this post hoc analysis, all available observations from the ACT‐RAY intent‐to‐treat (ITT) population were used to examine the occurrence of flares according to the second criterion proposed by van der Maas et al 2: increase in DAS28, between 2 visits, of >1.2 or >0.6 if the current DAS28 ≥3.2 (termed “DAS28‐2 flare”). Analyses were performed for the other proposed flare criteria (see Supplementary Table 1, available in the online version of this article at http://onlinelibrary.wiley.com/doi/10.1002/acr.22633/abstract). Event rates were reported as the count per 100 patient‐years, with evaluable followup, and time‐to‐event variables were assessed using the Kaplan‐Meier method.

In patients under constant treatment between week 24 and week 52, DAS28‐2 flares were assessed with respect to the previous assessment (reference visit), as well as sustained (consecutive) flares 2 and 3 visits after a given reference visit, up to week 44. The last observation carried forward method was used to impute missing DAS28 values prior to valid observations. For 28 patients (9%) who tapered oral corticosteroids during this study phase, observations were censored at the following visit.

In patients discontinuing TCZ in the second year, DAS28‐based flare was compared with investigator flare (reference) in 2 types of analyses. First, patients with an investigator flare were evaluated and assessed against the proportion with DAS28‐based flare at the same time. In a second analysis, 5 time points were selected after TCZ discontinuation (immediately after TCZ discontinuation and at 30, 60, 90, and 120 days thereafter) to evaluate the sensitivity and specificity of flare detection over time, based on DAS28‐2, using investigator flare as a reference, with the agreement between the 2 definitions based on Cohen's kappa.

Results

Patient disposition and background characteristics

The ITT study population included 553 patients enrolled at 118 centers in 19 countries from Europe, America, and Asia. The patients’ mean ± SD disease duration was 8.2 ± 8.2 years and their mean ± SD baseline DAS28‐ESR was 6.4 ± 1.0. At week 24, 313 patients had achieved LDA. In the second year, 238 patients discontinued TCZ after reaching sustained remission and were assessed for flares in the study (Figure 1).

Figure 1

Overview of patient analysis set. ITT = intent‐to‐treat; MTX = methotrexate; TCZ = tocilizumab.

DAS28‐2 flares under constant treatment

Disease activity between weeks 24 and 52 was stable (mean DAS28 of 2.24 and 2.21, respectively), and <1% of patients withdrew for efficacy reasons. Despite this, 136 DAS28‐2 flares per 100 patient‐years were recorded in 50% of patients (Kaplan‐Meier estimate) with a mean DAS28 of 3.7. Flares corresponded mostly to isolated disease activity peaks, followed by return to previous levels (Figure 2): the rate of sustained flares ≥2, and 3 visits after a given reference visit (visit of the first flare), up to week 44 was 52 and 25 per 100 patient‐years, concerning 17% and 8% of patients, respectively. Over the same period, 36% of patients had ≥1 DAS28‐2 flare. The flare incidence based on the other criteria varied quite widely, but the reduction in rates for sustained flares was similar to DAS28‐2 (see Supplementary Table 2, available in the online version of this article at http://onlinelibrary.wiley.com/doi/10.1002/acr.22633/abstract).

Figure 2

Flares corresponded mostly to isolated disease activity peaks followed by return to previous levels. For patients randomized to receive tocilizumab plus methotrexate (MTX) (A) or placebo (B), red dots indicate DAS28‐2 flares (Disease Activity Score in 28 joints between 2 visits >1.2, or >0.6 if the current DAS28 ≥3.2). The analysis is based on the constant treatment population (n = 313), i.e., patients with DAS28 <3.2 (horizontal broken line) at week 24.

Investigator flares versus DAS28‐2 flares after TCZ discontinuation

After discontinuation of one of the study drugs according to the protocol, most patients experienced an investigator flare (n = 200; Kaplan‐Meier estimate 85% [95% confidence interval (95% CI) 80%–89%]; median time to flare: 87 days [95% CI 85–113 days]), with the majority occurring following discontinuation of the first drug (TCZ) (see Supplementary Figure 1, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.22633/abstract). Mean values for disease activity measures at the time of TCZ discontinuation and at flare, respectively, were as follows: DAS28‐ESR 1.6 and 4.4; tender joint count 1.9 and 9.1; swollen joint count 0.64 and 4.6; Health Assessment Questionnaire Disability Index 0.44 and 0.87; and patient global assessment of disease activity (100‐mm visual analog scale) 13.4 mm and 34.6 mm. Of the patients with an investigator flare, 94% also had a DAS28‐2 flare at the same visit. For consecutive DAS28‐2 flares including the previous visit and at the 2 previous visits, the percentages were 52% and 35%, respectively. Similar patterns were observed for the other flare criteria (see Supplementary Table 1, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.22633/abstract).

DAS28‐based flares were detected faster and more frequently than investigator flares (median time: 56 versus 90 days). All patients with an investigator flare had a previous or concomitant DAS28‐2 flare. Comparison of the 2 definitions at the 5 pre‐specified time points (Table 1) demonstrated high sensitivity (88–100%), low specificity (57–68%), and modest agreement (Cohen's κ <0.41). The prevalence of investigator flare at each of these time points varied between 6.2% and 21.4%, with DAS28‐2 flares between 45–53%, or 1.7–7.3 times higher. The positive predictive value was low (14–42%) and the negative predictive value was high (96–100%). Analyses based on the other criteria led to similar conclusions (see Supplementary Table 3, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.22633/abstract).

Table 1

Investigator and DAS28‐2 flares in the cross‐sectional analyses following tocilizumab discontinuation in the second year: values from 2 × 2 tables (DAS28 flare as a predictor of investigator flare) at the visit following the indicated number of days after tocilizumab discontinuationa

	Day 0 (n = 210)	Day 30 (n = 170)	Day 60 (n = 131)	Day 90 (n = 100)	Day 120 (n = 80)
Prevalence of investigator flares	21.4 (45/210)	19.4 (33/170)	19.8 (26/131)	21.0 (21/100)	6.3 (5/80)
Prevalence of DAS28‐2 flares	47.1 (99/210)	51.2 (87/170)	49.6 (65/131)	53.0 (53/100)	45.0 (36/80)
Sensitivity	93.3 (42/45)	87.9 (29/33)	100 (26/26)	90.5 (19/21)	100 (5/5)
Specificity	65.5 (108/165)	57.7 (79/137)	62.9 (66/105)	57.0 (45/79)	41.3 (44/75)
Positive predictive value	42.4 (42/99)	33.3 (29/87)	40.0 (26/65)	35.8 (19/53)	13.9 (5/36)
Negative predictive value	97.3 (108/111)	95.2 (79/83)	100 (66/66)	95.7 (45/47)	100 (44/44)
Kappa	0.41	0.28	0.4	0.30	0.15
P†	< 0.0001	< 0.0001	< 0.0001	0.0001	0.0157

Values are the percentage (no./total no.) unless indicated otherwise. DAS28‐2 = Disease Activity Score in 28 joints between 2 visits >1.2 or >0.6 if the current DAS28 ≥3.2.

By Fisher's exact test.

Discussion

This study contributes to the discussion about defining flares in RA, as we analyzed data from a treatment strategy study with a 4‐weekly visit schedule. This schedule is more frequent than in most published studies, which rely on assessments every 3 months. In a study phase characterized by constant treatment, in stable disease activity and minimal efficacy‐related withdrawals, the DAS28‐based flare criteria from a published validation study 2 were met quite often; however, they were mostly in relation to disease activity fluctuations isolated to a single visit, potentially due to normal variability of clinical assessments. Recent modeling work supports the existence of high random variability in DAS28 over time 5. Therefore, utilizing a trend of DAS28 values over more than 1 visit, similar to the approach used by many investigators in the present study, may be more useful than single assessments in understanding disease progression. Similar incidences of flare have been observed in patients receiving constant treatment of anti‐tumor necrosis factor inhibitors 6.

After biologic agent discontinuation, investigator flares were diagnosed at fairly high disease activity levels, which was also the case for the patients’ assessments. Investigator judgment used to identify flares was more stringent than the DAS28‐based definitions, as indicated by a high sensitivity but rather low specificity. Understanding the correct time to respond to disease flares, without overtreating, requires further study.

In this analysis, we only used clinical disease activity for the assessment of flare; however, publications from the OMERACT RA Flare Group 1, 7 recently suggested including the patients’ perspective 8 and expanding the set of domains to, among others, fatigue, stiffness, and function. Also, in the IMPROVED study, patients who were treated for flares based on DAS28 criteria had similar efficacy outcomes to those who were treated based on nonprotocol‐endorsed criteria, demonstrating that DAS28‐based criteria to detect flares that matter for outcomes are limited 9.

This study used physician assessments to define flare in the context of a clinical trial, where the primary treatment goal was drug‐free remission. A limitation of this approach is the variability between physicians in their assessments and opinions of when treatment is warranted.

Investigator judgment used to identify flares was more restrictive than the best validated DAS28‐based definition, and solely basing treatment decision on these flare criteria may lead to unnecessary interventions. Clinical intervention may be more appropriate after flare is observed at consecutive visits. Indeed, the OMERACT RA Flare Group also recognized that duration and intensity are both important to define disease flares that impact physical function and quality of life.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Dougados had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design

Dougados, Huizinga, Aassi, Bernasconi.

Acquisition of data

Aassi.

Analysis and interpretation of data

Dougados, Huizinga, Choy, Bingham, Bernasconi.

ROLE OF THE STUDY SPONSOR

F. Hoffmann‐La Roche, Ltd. provided access to the ACT‐RAY database, statistical analysis, and meeting support of the researchers, and agreed to run this post hoc analysis of the ACT‐RAY trial. Publication of this article was not contingent upon approval by F. Hoffmann‐La Roche, Ltd.

ADDITIONAL DISCLOSURE

Dr. Aassi is an employee of Roche Global Medical Affairs.

Supplementary Table 1: Percentage of DAS28‐based flares among patients with investigator flare following tocilizumab discontinuation

Supplementary Table 2: Incidence of flares (at ≥1 visit) and sustained flares (≥2 or 3 consecutive visits) over the week 28 to week 52 constant treatment period according to the different criteria

Supplementary Table 3: Diagnostic test characteristics for the prediction of investigator flares: mean values from 2 × 2 tables based on cross‐sectional analyses at day 30 (N=170), day 60 (N=131), and day 90 (N=100) following tocilizumab discontinuation for the different flare criteria

Supplementary Figure 1 Kaplan‐Meier plot of time to investigator‐flare after protocol‐driven discontinuation of tocilizumab. The dotted line represents an estimate where observations are censored (“+”) at the time of discontinuation of a further drug (MTX/placebo or, in a minority of patients, an open‐label DMARD).

Click here for additional data file.