Impact of rheumatoid arthritis disease activity test on clinical practice.

BACKGROUND: Variability exists in the assessment of disease activity in rheumatoid arthritis (RA) patients that may affect quality of care.
OBJECTIVES: To measure the impact on quality of care of a Multi-Biomarker Disease Activity (MBDA) test that quantitatively assesses RA disease activity.
METHODS: Board-certified rheumatologists without prior experience with the MBDA test (N = 81) were randomized into an intervention or control group as part of a longitudinal randomized-control study. All physicians were asked to care for three simulated RA patients, using Clinical Performance and Value (CPV™) vignettes, in a before and after design. CPV™ vignettes have been validated to assess the quality of clinical practice and identify variation in care. The vignettes covered all domains of a regular patient visit; scores were determined as a percentage of explicit predefined criteria completed. Three vignettes, representing typical RA cases, were administered each round. In the first round, no physician received information about the MBDA test. In the second round, only physicians in the intervention group were given educational materials about the test and hypothetical test results for each of the simulated patients. The outcome measures were the overall quality of care, disease assessment and treatment.
RESULTS: The overall quality scores in the intervention group improved by 3 percent (p = 0.02) post-intervention compared with baseline, versus no change in the control group. The greatest benefit in the intervention group was to the quality of disease activity assessment and treatment decisions, which improved by 12 percent (p<0.01) compared with no significant change in the control group. The intervention was associated with more appropriate use of biologic and/or combination DMARDs in the co-morbidity case type (p<0.01).
CONCLUSIONS: Based on these results, use of the MBDA test improved the assessment and treatment decisions for simulated cases of RA and may prove useful for rheumatologists in clinical practice.

RCT Entities:   Population Interventions Outcomes

Introduction

Rheumatoid arthritis (RA) is a chronic, systemic inflammatory disorder that affects approximately 1.5 million people in the United States.[1] Unlike disease areas such as cardiology that have extensive research and appropriate use criteria, evidence-based medicine for RA diagnosis, evaluation and treatment are based on largely subjective criteria.[2], [3] Until 2010, the classification criteria for RA had not changed in 23 years.[4] Recent treatment guidelines direct rheumatologists to treat RA promptly and aggressively, aiming for remission as a therapeutic target.[5], [6] Nevertheless, early recognition of signs and symptoms and timely referral to rheumatologists remain challenging; appropriate use of non-biologic and biologic disease-modifying anti-rheumatic drugs (DMARDs) significantly improves outcomes but requires close monitoring of disease activity, including laboratory evaluations and imaging that is often difficult to achieve.[3], [7] A number of composite tools are available for assessing disease activity in RA, of which 6 have been recommended by the American College of Rheumatology (ACR).[8] They are infrequently used in practice and have limitations, including reliance on subjective measures with frequent discordance between physician and patient assessments.[9] Patient global assessment and its contribution to the ACR/European League Against Rheumatism remission criteria are affected by non-inflammatory factors.[10] Moreover, individual variability in therapeutic responses and lack of objective biomarkers for identification of active disease can lead to variability in patient care that may affect the management of RA [11]–[15].

New technology and diagnostic tools have been shown to lower variability in clinical areas such as cardiology, radiology and oncology, [16] but so far this effect has not been demonstrated in rheumatology. Reducing the variability in rheumatologic assessments and treatment decisions would be beneficial in RA not only because it would lead to higher quality for the patient but it could lower aggregate health care costs. [17], [18] By objectively quantifying disease activity in RA, physicians may be better equipped to allocate resources accurately, avoid ordering unnecessary tests, and change therapy effectively [19].

A multi-biomarker disease activity (MBDA) blood test (Vectra DA™; Crescendo Bioscience, Inc., South San Francisco, CA) is currently available in the U.S. to assess disease activity in adult patients with RA. The test is done by taking serum prepared from patient peripheral blood and sending it to a central laboratory for automated, multiplexed measurement of 12 protein biomarkers that play key roles in the underlying pathophysiology of RA. The MBDA test is based on an algorithm that uses the concentrations of the 12 biomarkers to generate a score that represents the level of RA disease activity on a scale of 1 (lowest activity) to 100 (greatest)[20], [21] and, in studies of analytical validation, the test is precise and reproducible.[20] The MBDA test was developed to correlate with the 28-joint Disease Activity Score (DAS28) and has been clinically validated by correlations with DAS28 and other disease activity measures in independent RA cohorts, with thresholds established for low, moderate and high disease activity.[22], [23] Other studies show that the MBDA test tracks responses to treatment with biologic and non-biologic DMARDs,[22], [23] and may potentially be an indicator of progressive joint damage in patients with RA.[24] The MBDA test is not validated for diagnosing RA.

There are currently no blood tests that are specific for RA. The two clinical laboratory tests most closely associated with RA, rheumatoid factor and anti-citrullinated protein antibody tests, are for diagnosis and prognosis. The only blood tests that are routinely used to assess disease activity in RA are the erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP). Both have limitations, as their values are in the normal range in up to half of patients with active disease, and they are often discordant with each other.[25], [26] Roles for the MBDA test in RA patient management include situations where clinical evaluations and laboratory test results provide a mixed picture of disease activity, or assessment is confounded by symptoms from diseases other than RA. The MBDA test is well suited to being studied with case simulations because, as an objective test, the inclusion of a hypothetical MBDA score in a case simulation is sufficient to duplicate its real-life inclusion in the clinical decision-making process. Thus to evaluate the potential clinical utility of the MBDA test, we used simulated cases of RA patients and hypothetical MBDA scores to evaluate clinical practice by rheumatologists before and after exposure to the MBDA test.

Methods

Overview

This is a longitudinal randomized controlled study of practicing rheumatologists from across the U.S. with no prior experience with the MBDA test. In a “before and after” design, rheumatologists were asked to care for simulated patients with RA, using Clinical Performance and Value (CPV™) vignettes via web-based interactive sessions. Physicians completed the vignettes at baseline (3 cases pre-intervention, Round 1) and again 12 weeks later (3 cases post-intervention, Round 2). In Round 1, none of the rheumatologists received any information about the MBDA test and none of the vignettes included MBDA scores. In Round 2, rheumatologists who were randomized to the intervention group were introduced to the MBDA test and provided with a hypothetical test result for each Round 2 vignette, whereas MBDA test information and results were not made available to physicians randomized to the control group. This study was conducted in accordance with the ethical standards of the institutional review board (IRB), approved by the Essex IRB, Lebanon, NJ. Participants provided written consent to participate, and Essex IRB approved the consent form.

The Intervention

The intervention group received hypothetical MBDA scores, on a scale from 1 to 100, for each patient in the Round 2 vignettes. As is true for the actual MBDA test, lower scores indicated less disease activity and higher scores generally indicated a need to initiate or change therapy. Prior to receiving hypothetical MBDA scores rheumatologists in the intervention group were provided with information about the MBDA test and offered a variety of educational modalities, including emailed materials, mailed materials, access to a podcast, invitation to a live webinar and/or access to the Principal Investigator of the study and the Chief Medical Officer of Crescendo Bioscience for answers to specific questions.

Eligibility and Selection of Physicians

Eligibility for participation in the study was assessed using a physician questionnaire. Physicians had to be (1) board-certified rheumatologists, (2) English-speaking, (3) practicing in a community/non-academic based setting, (4) in practice with >200 RA patients in their care, (5) accessible by Internet, and (6) inexperienced with the MBDA test. From a nationally representative list of 2600 eligible rheumatologists, 81 physicians were selected for participation in this study (Figure 1). These physicians were randomized into one of two arms: 37 in the intervention arm, and 44 in the control arm. Only one physician was lost to follow-up between baseline and post-intervention assessments.

Figure 1

Flow diagram of sample selection.

Clinical Performance and Value™ vignettes

The quality of clinical performance was measured at baseline (Round 1) and after 12-weeks (Round 2) using Clinical Performance and Value (CPV™) vignettes. CPV™ vignettes have been demonstrated to be a validated means for assessing differences in clinical practice and inherent variation in care independent of case mix.[27]–[29] The vignettes used in this study simulated clinical encounters involving patients with RA. In each vignette, the rheumatologists provided open-ended responses regarding clinical care. Responses were scored in 5 domains: taking a medical history, performing a physical examination, ordering appropriate laboratory and imaging tests, assessing disease activity and physical function, and prescribing treatment. Scoring criteria were derived from a literature review and expert opinion to generate an explicit evidence base. Scores for the completed vignettes were the percentage of physician answers that matched these evidence-based criteria.

Three types of CPV™ vignettes, representing typical cases seen by practicing rheumatologists, were prepared to evaluate the impact of the MBDA test result on patient management. Four vignettes were prepared for each type. Type A vignettes comprised RA cases insufficiently controlled with treatment and requiring addition of or increase in dosage of a non-biologic DMARD; Type B vignettes included RA cases who received inadequate treatment judged to require addition of a non-biologic or biologic DMARD; and Type C vignettes described cases of adequate treatment for RA but worsening symptoms due to one of four co-morbidities (depression, fibromyalgia, adenocarcinoma metastatic to bone and osteoarthritis). All physicians completed 2 randomly assigned cases for each of the 3 case types, one at baseline and one 12 weeks later or after the intervention, for a total of 6 vignettes per physician. Physicians were surveyed at the end of each round after completing 3 cases and asked what percentage of their cases the vignettes represented. On average the 3 case types in this study represented 62% to 80% of the types of patients that the rheumatologists in this study saw in their practices.

Analysis

Our primary hypothesis was that the MBDA test would improve quality of care. Quality was measured by the CPV™ vignettes. We used the overall quality score and in subsequent analyses determined that the disease activity assessment and treatment quality domain scores were the domains most affected by introduction of the MBDA test, as might be expected. As a corollary, we further hypothesized that quality effects would occur across all vignette case types (Types A, B, and C). Secondary outcomes included appropriate use of non-biologic or biologic DMARDs or combinations, number of imaging tests ordered, number of laboratory tests ordered, and use of other resources (which included use of office visits).

We used multivariate regression modeling to compare outcomes in Round 2 versus Round 1 in the intervention and the control groups. For the primary outcomes ― overall CPV™ quality scores and scores for the combined disease assessment and treatment domains–we used linear regression models of continuous data. Regression coefficients in the model included dummy variables for (1) intervention versus control groups, which captured whether the intervention group was significantly different from controls at baseline, (2) intervention group scores at Round 2 versus baseline, which captured changes over time in the intervention group, (3) control scores at Round 2 versus baseline, which captured changes over time in the control group, and (4) physician characteristics identified a priori as potential confounders: age, gender, years of rheumatology practice, number of days worked per week, practice location, single or multispecialty practice, employed by a health system, use/employ mid-level practitioners in office, on-site phlebotomy, number of patients seen per week and number of RA patients, and insurance coverage of patients (percent under Medicaid and percent under Medicare). We also included the interaction between facility variables: mid-level practitioners and multi-specialty facility.

For the secondary outcome, appropriate use of biologic or combination DMARDs, we used logistic regression controlling for the same variables as described above. We used a Poisson distribution for the number of imaging tests ordered, number of laboratory tests ordered, and use of resources. Regression models of the secondary outcomes controlled for the same co-variates described above for the primary outcomes. All analyses were conducted using STATA 12.0.

Results

There were no significant differences in physician characteristics between the intervention and control groups at baseline, with the exception of a difference in mid-level providers at the facility (Table 1). Quality scores, measured by CPV™ overall and disease activity assessment and treatment domain scores, differed significantly at baseline between the three case types, which reflected the different clinical scenarios completed by rheumatologists (Table 2). Type C cases, which presented patients with potentially confounding co-morbidities, resulted in the lowest baseline scores.

Table 1

Baseline Characteristics of Intervention and Control group physicians, All Case types.

	Intervention (n = 37)	Control (n = 44)	p-value
Number of rheumatology cases seen ≥30 per week	48.65	62.79	0.21
% of Medicare patients	40.03	38.63	0.61
% of Medicaid patients	7.86	7.51	0.88
On-site Phlebotomy	78.38	74.42	0.68
Employed by a system	21.62	20.93	0.94
Male	75.68	67.44	0.42
Practice at least 5/days per week	70.27	58.14	0.27
Age	49.70	48.95	0.74
Facility is multi-specialty	48.65	55.81	0.53
Facility has midlevel providers	35.14	6.98	<0.001

Table 2

Baseline Mean CPV Overall and Disease Assessment and Treatment Quality Scores for All Study Physicians.

	Mean	Standard deviation	p-value
			(Relative to Type C)
Overall CPV Score
All Types	60.36	12.11	n/a
CPV Type A	61.97	12.48	<0.001
CPV Type B	62.01	11.69	<0.001
CPV Type C	57.09	11.58	n/a
CPV Disease Assessment and Treatment Domain Score
All Types	54.01	21.71	n/a
CPV Type A	61.73	21.98	<0.001
CPV Type B	53.25	20.88	0.03
CPV Type C	47.04	19.86	n/a

CPV indicates Clinical Performance and Value.

When we evaluated the impact of introducing the MBDA test into clinical practice on disease assessment and treatment quality scores, rheumatologists in the intervention group improved their quality scores by 12 percentage points for the 3 case types combined, after adjustment for the other covariates (p<0.001). This improvement was also significant for each of the 3 case types individually. The regression results, using combined CPV™ disease activity assessment and treatment domain scores, for all cases and disaggregated, are shown in Table 3. As expected, in the control group, no statistically significant improvements in treatment decisions were evident between Rounds 1 and 2.

Table 3

Multiple Linear Regression Models of CPV Disease Assessment and Treatment Scores by Case Type for All Cases and for Each Case Type.

	All Case Types		Type A		Type B		Type C
	Coef	p-value	Coef	p-value	coef	p-value	coef	p-value
MBDA test Intervention at Round 2 compared with Baseline	12.49	<0.001	13.06	0.01	11.56	0.02	14.23	<0.001
Control at Round 2 compared with Baseline	3.02	0.24	0.06	0.99	1.55	0.73	7.64	0.08
MBDA test Intervention compared with control at Baseline	−4.68	0.09	−6.24	0.22	−6.15	0.22	−1.87	0.70
Number of rheum cases seen ≥30 per week	1.55	0.48	2.77	0.49	−0.03	0.99	2.79	0.47
% of Medicare patients	0.04	0.67	0.02	0.89	0.16	0.30	−0.09	0.55
% of Medicaid patients	−0.23	0.03	−0.35	0.08	−0.14	0.45	−0.30	0.11
On-site Phlebotomy	−5.19	0.03	−4.16	0.33	−9.19	0.03	−3.05	0.46
Employed by a system	7.82	<0.001	12.66	<0.001	2.53	0.54	7.43	0.07
Male	0.62	0.79	0.56	0.89	4.00	0.32	−1.87	0.64
Practice at least 5 days per week	−1.05	0.62	−1.22	0.75	−3.01	0.42	1.71	0.65
Age	−0.16	0.14	−0.27	0.17	−0.04	0.86	−0.12	0.52
Facility is multi-specialty	3.37	0.13	5.81	0.15	0.21	0.96	4.54	0.25
Facility has midlevel providers	0.36	0.93	8.10	0.29	−1.89	0.81	−5.51	0.46
Facility has midlevel providers and is multi-specialty	−6.65	0.20	−12.07	0.20	−3.37	0.72	−3.65	0.69
Constant	66.05	<0.001	76.32	<0.001	60.37	<0.001	58.94	<0.001

CPV indicates Clinical Performance and Value; MBDA, multi-biomarker disease activity.

When we looked at the impact on the overall quality scores, there was also a statistically significant improvement in the intervention group compared with controls following use of the MBDA test (Table 4). Physicians in the intervention group improved their scores by 3 percentage points overall in the 3 case types combined, after adjustment for the other covariates. These effects did not differ by case type (regression analysis results not shown). In the control group there were no significant improvements between Rounds 1 and 2.

Table 4

Multiple Linear Regression Model of Overall CPV scores for all Case Types.

	Coef	p-value
MBDA test Intervention at Round 2 compared to Baseline	3.51	0.02
Control at Round 2 compared to Baseline	1.97	0.17
MBDA test Intervention compared to control at Baseline	−0.36	0.82
Number of rheum cases seen> = 30 per week	1.06	0.40
% of Medicare patients	−0.05	0.31
% of Medicaid patients	−0.26	<0.001
On-site Phlebotomy	−3.65	0.01
Employed by a system	4.39	<0.001
Male	−0.17	0.89
Practice at least 5/days per week	−1.97	0.10
Age	−.0001	.999
Facility is multi-specialty	3.36	0.01
Facility has midlevel providers	−0.84	0.73
Facility has midlevel providers and is multi-specialty	−5.91	0.05
Constant	66.57	<0.001

CPV indicates Clinical Performance and Value; MBDA, multi-biomarker disease activity.

Analysis of covariates controlled for in the quality models revealed some associations: employment in a health system resulted in a score that was greater by 8 percentage points for the combined disease activity assessment and treatment domain, and by 4 percentage points for the overall score. Analysis of overall scores revealed they decreased slightly (by 0.26 percentage point) with each percent increase in the portion of Medicaid patients in a practice. Outcome scores for rheumatologists employed by a health care system and for those working in multi-specialty practices were significantly greater, by 4 and 3 percentage points respectively. For disease activity assessment and treatment quality outcomes, the effect of working in a multi-specialty practice was similar (3 percentage points) and an even greater effect (8 percentage points) was observed for rheumatologists working in a health care system (Table 3).

Analysis of the secondary endpoints revealed that in the intervention group there was significant improvement in the appropriate use of biologic or combination DMARDs in patients with co-morbidities; however, this association was only evident in case Type C, for which no change in RA therapy was appropriate, and not the others (p = 0.008). The significant marginal effects for the MBDA test here indicated that rheumatologists in the intervention group were 10 percentage points more likely to appropriately use biologic or combination DMARDS compared with baseline. There was no significant change in the likelihood of correctly using biologics in the other two case types or in the control group over time. There was also no observable impact of use of the MBDA test in the models examining imaging, laboratory use, and referrals (combined resource use), either for case types combined or individually.

Discussion

This study examined the impact of introducing an MBDA test on the quality of clinical practice for simulated patients with RA. Findings indicate that having knowledge of and access to the MBDA test result translated into better assessment and treatment decision-making by rheumatologists, evidenced by improved patient vignette scores in the intervention group compared with controls. These results, observed in a randomized controlled design, support the findings of earlier studies indicating that offering physicians a quantitative measure of disease activity improves their practice and may lead to better clinical outcomes.[19], [30] In RA in particular, the lack of more objective measures of disease activity can make it difficult for physicians to make the most appropriate treatment decisions. The effect of the MBDA test intervention here indicates that by using a fully objective indicator of disease activity, rheumatologists may be able to more effectively assess and treat a variety of typical RA cases.

Poor quality and clinical variation are a challenge in general practice and specialty care.[31] New technology, however, is often not considered in discussions about how to improve practice where, increasingly, the health care conversation has turned to new payment models, performance incentives and care reorganization. However, advances in technology that provide physicians with information that cannot be derived from the physical exam or from a non-specific laboratory test, such as ESR or CRP, may help with decisions of whether to maintain or alter treatment. In this study the MBDA test led to significant improvement in quality for the combined domain of clinical assessment and treatment, where benefit is the most plausible, compared with the other domains that were evaluated–taking a medical history, performing a physical examination, and ordering appropriate laboratory and imaging tests. Thus, introduction of the MBDA test helped eliminate clinical and therapeutic ambiguity, resulted in better treatment decisions and improved quality of care for the RA cases represented in the vignettes.

These results further indicate that the MBDA test was particularly useful in those cases where there was an underlying co-morbidity, evident by the significant estimate of effects of the intervention on disease assessment and treatment domain quality scores in the type C vignettes where there was significant improvement in the appropriate use of biologic or combination DMARDs. The MBDA test thus appears to have helped rheumatologists distinguish between symptoms of RA and a co-morbid condition, and thereby avoid changing RA therapy when the new complaint was due to a different disease.

There are limitations to this study. First, our sample size was relatively small and thus likely limited our statistical power for the secondary outcomes. Second, because the control group received neither MBDA training, nor MBDA test results, the study did not directly address the independent effect of training alone. It seems likely, however, that little or no effect should have resulted from training independent of the MBDA test result. The study did not analyze the long-term impact of use of the MBDA test on physician practice and resulting improvements in patient outcomes; a third round of vignette administration could demonstrate whether the improvements observed were sustained beyond the limited study period. In addition, the MBDA test potentially may be used for long-term disease tracking, and this assessment of its use at a single time point, may not fully assess its impact on routine practice. Future studies could be designed to overcome these limitations by using a larger sample and longer time frame. Future work might also include additional case types, patient outcomes linked to the two groups of rheumatologists, sampling of academic practices and more analysis on resource utilization. Importantly, although the MBDA test results used in this study were designed to represent a real-world test, these results were hypothetical and might differ from those obtained for actual patients.

The methodology used in this study may prove particularly helpful to others wanting to demonstrate clinical effectiveness. Traditional clinical effectiveness trials are costly, confounded by case mix variation among different sites, and typically exceed the budgets of entrepreneurs, potentially stifling innovation. [32] By comparison, we used six CPV™ vignettes, expressly written for this study in RA, representing three of the most common types of RA cases: patients inadequately treated with non-biologic DMARDs, patients inadequately treated with a non-biologic DMARD in combination with another non-biologic or a biologic DMARD, and patients with co-morbidities. The case simulations used in this study have been rigorously validated against standardized patients in two large studies and shown to match actual practice.[28], [29] Moreover, research using this evaluation technology has shown that higher quality vignette scores reflect better patient care.[27], [33], [34]

Expanding physician knowledge about critical details of the patient's disease not only facilitates clinical decision-making but may also optimize resource allocation by matching the best treatment to the individual patient type. It is not clear what impact this will have on costs. In RA, better precision in assessing disease activity may result in more appropriate use of non-biologic and/or biologic DMARDs, thereby reducing cost and risk of significant adverse effects that may be associated with biologic DMARDs. In other cases, better precision in assessing disease activity may indicate an earlier need for biologic DMARDs, which in turn can improve long-term outcomes, including lowering costs.

In spite of the limitations, this study shows that the availability of an MBDA test in clinical practice may impact decision-making and improve quality of care for patients with RA. Novel technologies for assisting physicians to assess RA disease activity and more appropriately treat patients should be similarly evaluated for gauging the impact on clinical care.