High Adherence to System-Level Performance Measures for Rheumatoid Arthritis in a National Early Arthritis Cohort Over Eight Years.

OBJECTIVE: To assess adherence to 3 system-level performance measures in a national early rheumatoid arthritis (RA) cohort.
METHODS: Patients enrolled in the Canadian Early Arthritis Cohort (2007-2015) who met 1987 or 2010 American College of Rheumatology/European League Against Rheumatism criteria with <1 year of symptom duration and ≥1 year of followup after enrollment were included. Performance measures assessed were the percentage of RA patients seen in yearly followup, and the number of gaps between visits of >12 or >14 months, the percentage of RA patients treated with a disease-modifying antirheumatic drug (DMARD), and days from RA diagnosis to initiation of a DMARD. Results are shown stratified by enrollment year to assess for temporal changes in performance.
RESULTS: A total of 1,763 early RA patients were included (mean age 54 years, 73% female, and 82% white). At enrollment, mean ± SD disease duration was 6 ± 3 months, and Disease Activity Score in 28 joints was 5.1 ± 1.5. Over 8 years, the proportion of patients seen in annual followup declined from 100% to 91%. Over followup, 42% of patients had 0 gaps in care of >12 months, and 64% had 0 gaps >14 months. The percentage of DMARD-treated early RA patients was and remained high (95-87%), and the percentage receiving DMARDs within 14 days of diagnosis was 75%. Median time-to-DMARD therapy was 1 day, indicating DMARDs were initiated at diagnosis (90th percentile 93 days).
CONCLUSION: There was evidence of high adherence to system-level performance measures in this early RA cohort following a protocol. Small declines in performance were noted with increasing length of patient followup. Our findings are useful for performance measure benchmarking.

Introduction

Measuring health care quality has received increasing attention over the last 2 decades since landmark reports from the Institute of Medicine described important gaps in health care 1, 2. In rheumatology, organizations such as the American College of Rheumatology (ACR) in the US and the National Institute for Health Care Excellence (NICE) in the UK have developed quality indicators 3, 4 and quality standards 5 as tools for monitoring and improving the quality of care provided to patients with rheumatoid arthritis (RA).

There was evidence of a high level of adherence to performance measures for early treatment and followup in early rheumatoid arthritis (RA) patients from rheumatology clinics across Canada who were enrolled in a prospectively followed cohort.

Measuring adherence to performance measures using an established early RA cohort is feasible and may be used for benchmarking in other settings.

The Arthritis Alliance of Canada (AAC) is a not‐for‐profit organization bringing together arthritis stakeholder groups from across Canada to participate in projects to improve the lives of patients living with arthritis. The AAC has developed an approach to models of care for inflammatory arthritis 6 and a framework for model‐of‐care evaluation that promotes provision of high‐quality care. As part of this evaluation framework, a set of 6 system‐level performance measures for inflammatory arthritis care capturing early access to care and treatment for inflammatory arthritis patients were developed 7. The measures capture the following concepts: waiting times for rheumatologist consultation, percentage of patients seen by a rheumatologist, percentage of patients seen in annual followup, percentage of patients treated with a disease‐modifying antirheumatic drug (DMARD), time‐to‐DMARD therapy, and rheumatologists per capita 7. To date, the measures have not been tested in regular rheumatology practice settings, as quality of rheumatology care is not routinely monitored in Canada outside of the context of research.

The objective of the present study is to report on 3 of the system‐level performance measures (percent seen in yearly followup, percent taking a DMARD, and time‐to‐DMARD start) in early RA patients enrolled in a pan‐Canadian study (the Canadian Early Arthritis Cohort [CATCH]) 8. The remainder of the measures were not evaluated, as the data were not available and/or the measure was not appropriate to measure in CATCH (e.g., waiting times were not captured in CATCH, rheumatologists per capita has been evaluated elsewhere 9, and the percentage of patients seen by a rheumatologist is a measure captured using population administrative data).

Patients and methods

CATCH is a prospective observational study of patients with early RA seen in academic and community rheumatology clinics across Canada since its inception in January 2007 8. Patients in CATCH are recruited by a member of their health care team and receive usual care and treatment at the discretion of their rheumatologist, but sites are encouraged to target treatment to attain remission. Members of the CATCH site primary investigators are shown in Appendix A. Patients are followed according to a standard protocol with assessments at baseline (at enrollment) and every 3 months for the first year, every 6 months, in year 2, and annually thereafter. Data from visits outside of these standard assessments are not captured in CATCH. At each visit a standardized assessment is completed, including (but not limited to) measurement of disease activity, evaluation of medications, and patient‐completed validated questionnaires on symptoms and physical function. Standard‐of‐care laboratory tests are drawn for each protocol visit at the discretion of the provider 8.

Inclusion criteria

Patients from CATCH were included in the study if they met either 1987 ACR 10 or 2010 ACR/European League Against Rheumatism (EULAR) criteria for RA with <1 year of symptom duration at enrollment 11. Patients who had not yet reached their expected 1‐year followup date or those missing the date of first treatment start were also excluded, as they would not have enough followup time to be eligible for performance measurement. Followup time began upon enrollment on or after January 1, 2007 and continued until patient withdrawal from the study or the end of the study period (May 9, 2016).

Analysis

Descriptive statistics were calculated for baseline characteristics. Chi‐square tests were used to examine differences in performance over time. Control charts were created for the measures depicting performance over time compared to overall mean and with upper and lower control limits set at 3 SDs above and below the mean. Three system‐level performance measures were operationalized and reported.

Percentage of patients seen in yearly followup by rheumatologists

The percentage was calculated using fixed 12‐ and 14‐month windows. Each patient's followup time began at enrollment and continued until the patient withdrew from the study, was lost to followup, or the study observation period ended, whichever came first. The proportion of patients meeting this measure was calculated using the denominator (e.g., expected visits: 1 for each calendar year) and the observed visits (if they had at least 1 during the year) as the numerator. The number of gaps of >12 and >14 months duration (to allow for a lag‐time in appointment scheduling) between consecutive visits was also calculated and reported as a proportion of the total number of years of followup for all patients in the cohort.

Percentage of patients taking a DMARD or biologic DMARD during the measurement year

The percentage of patients using DMARDs was calculated between the enrollment date and the end of followup on a yearly basis for each patient. The denominator included the number of patients in the cohort during the measurement year, and the numerator included the number of patients with at least 1 DMARD or biologic DMARD prescription during the year, as documented in the patient record.

Time‐to‐DMARD therapy

Time was calculated by measuring the time between the physician‐reported date of RA diagnosis and the date the first DMARD was started, and was reported as the median and 90th percentiles. Additionally, the percentage of patients treated within the Wait Time Alliance 12 benchmark of 14 days was also reported. For this outcome, patients with a missing diagnosis date were excluded. The DMARDs/biologic DMARDs included in this measure are listed in Supplementary Appendix A (available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract).

The measures are reported yearly between 2007 and 2015, with the exception of time‐to‐DMARD therapy, which is reported until 2014, as a complete year of followup was necessary to determine time‐to‐DMARD therapy, and individuals with a diagnosis in 2015 may not have had a complete year of followup in 2016 to meet this measure. All results were also stratified by year of enrollment in the cohort.

Ethics

The present study was approved by the University of Calgary Research Ethics Board (REB15‐2271). The CATCH study was approved by local research ethics boards at each center, and all participants provided written informed consent.

Results

A total of 1,763 early RA patients met inclusion criteria for this study (Figure 1). Enrollment characteristics were similar to other early RA cohorts, as patients had a mean age of 54 years and 73% were female (Table 1). The mean ± SD disease duration at enrollment was 6 ± 3 months and the median followup was 4 years (range 1–9 years).

Figure 1

Inclusion criteria for the study. There were 1,763 Canadian Early Arthritis Cohort (CATCH) patients included in the analytic sample at baseline (final box in the figure). Over time there was some attrition due to withdrawals or patients who were lost to followup: 11 in the first year, 68 between the first and second years, 65 between the second and third years, 59 between the third and fourth years, 50 between the fourth and fifth years, and 65 at ≥5 years. ACR = American College of Rheumatology; EULAR = European League Against Rheumatism; RA = rheumatoid arthritis; DMARD = disease‐modifying antirheumatic drug.

Table 1

Baseline characteristics of patients included in the study from the Canadian Early Arthritis Cohorta

Characteristics	Total sample (n = 1,763)
Age, mean ± SD years	54 ± 15
Female	1,283 (73)
Disease duration, mean ± SD months	5.7 ± 3.0
Meeting ACR/EULAR 2010 classification criteria	1,575 (89)
Meeting 1987 ACR classification criteria	1,235 (71)
DAS28‐ESR, mean ± SD	5.1 ± 1.5
TJC68, mean ± SD	13 ± 9
TJC28, mean ± SD	9 ± 7
SJC66, mean ± SD	10 ± 8
SJC28, mean ± SD	8 ± 6
Physician global assessment, mean ± SD	4.9 ± 2.5
Patient global assessment, mean ± SD	5.8 ± 2.9
HAQ DI, mean ± SD	1.1 ± 0.7
Anti‐CCP positive	750 (43% overall or 60% of nonmissing)b
RF positive	1,075 (61)
Smoking
Never	741 (42)
Current	315 (18)
Former	699 (40)
Education greater than high school	906 (51)
Baseline treatmentc
Nonbiologic DMARDs	1,380 (78)
Biologic DMARDs	40 (2)
Small molecule inhibitors or other immunosuppressive agents	0 (0)
Treatment over course of followupc
Nonbiologic DMARDs	1,619 (92)
Biologic DMARDs	342 (19)
Small molecule inhibitors or other immunosuppressive agents	6 (0.3)

Values are the number (%) unless indicated otherwise. ACR = American College of Rheumatology; EULAR = European League Against Rheumatism; DAS28‐ESR = Disease Activity Score in 28 joints using the erythrocyte sedimentation rate; TJC = tender joint count; SJC = swollen joint count; HAQ DI = Health Assessment Questionnaire disability index; anti‐CCP = anti–cyclic citrullinated peptide; RF = rheumatoid factor; DMARDs = disease‐modifying antirheumatic drugs;

There are 511 missing.

Treatments included in these categories are shown in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract. Due to low numbers, the numbers of patients taking small‐molecule inhibitors or other immunosuppressive agents have been combined for reporting.

The percentage of patients seen in yearly followup is shown in Table 2 (control chart available in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). The overall percentage seen in yearly followup declined from 100% in 2008 to 85% in 2015 (P = 0.24). However, when analyses were stratified by year of enrollment, this trend was explained mostly due to an increasing proportion of patients with longer cohort followup in later years (e.g., more people in later years with longer followup who had a greater opportunity to get lost to followup over time). There were few statistically significant differences in followup rates when the stratified sample was reviewed, with the exception of the patients enrolled in 2011, who had higher rates of missed followups over time.

Table 2

CATCH patients seen in yearly followup, stratified by enrollment yeara

	2008	2009	2010	2011	2012	2013	2014	2015	P
Numeratorb	91	338	581	756	933	1,023	1,060	1,056	–
Denominatorc	91	352	619	833	1,029	1,168	1,239	1,238	–
Proportion, %	100	96	94	91	91	88	86	85	0.24
Enrollmentd
2007	91 (100)	80 (89)	66 (83)	53 (79)	50 (76)	45 (79)	37 (77)	39 (91)	0.74
2008	–	258 (98)	228 (91)	183 (83)	176 (86)	154 (81)	134 (80)	113 (84)	0.38
2009	–	–	287 (99)	253 (93)	227 (87)	202 (84)	187 (88)	152 (83)	0.40
2010	–	–	–	267 (98)	239 (93)	199 (84)	173 (80)	162 (84)	0.19
2011	–	–	–	–	241 (99)	210 (92)	151 (75)	132 (74)	0.01e
2012	–	–	–	–	–	213 (99)	190 (92)	148 (83)	0.26
2013	–	–	–	–	–	–	188 (99)	158 (92)	0.45
2014	–	–	–	–	–	–	–	152 (97)	–

Values are the number (%) unless indicated otherwise. CATCH = Canadian Early Arthritis Cohort.

The numerator represents the total number of patients in CATCH seen yearly in the cohort.

The denominator represents the total number of patients in CATCH with an expected followup in each calendar year, excluding patients who withdrew from followup during the calendar year.

Patients meeting the followup measure, stratified by year of enrollment.

Statistically significant.

Similarly, the number of gaps in care also increased with increasing length of followup (Table 3), with the most common pattern of care being no gaps of >12 months between years 1 and 3 of followup, but by year 4, 50% of patients had at least 1 gap, and by year 6 over 50% had 2 or more gaps in care. When a 14‐month window was used, over half of the patients had no gaps until year 5 of followup, with increasing numbers of gaps seen after 5 years (Table 3). Overall, using a 14‐month window, 64% of patients had no gaps in care over followup compared to 42% using a 12‐month window.

Table 3

Gaps in care of >12 and >14 months in Canadian Early Arthritis Cohort patients followed for up to 9 yearsa

	Number of gaps >12 months				Number of gaps >14 months
	0	1	2	≥3	0	1	2	≥3
Total sample	747 (42)	479 (27)	356 (20)	181 (10)	1,136 (64)	485 (28)	123 (7)	19 (1)
Years in cohort (no.)
1 (172)	165 (96)	7 (4)	0	0	170 (99)	2 (1)	0	0
2 (268)	241 (90)	25 (9)	2 (1)	0	249 (93)	18 (7)	1 (<1)	0
3 (270)	187 (69)	78 (29)	5 (2)	0	232 (86)	35 (13)	3 (1)	0
4 (243)	90 (37)	122 (50)	27 (11)	4 (2)	162 (67)	75 (31)	6 (2)	0
5 (242)	38 (16)	111 (46)	78 (32)	15 (6)	124 (51)	104 (43)	13 (5)	1 (<1)
6 (242)	17 (7)	81 (33)	112 (46)	32 (13)	91 (38)	110 (45)	35 (14)	6 (2)
7 (175)	5 (3)	37 (21)	78 (45)	55 (31)	56 (32)	83 (47)	33 (19)	3 (2)
8 (118)	4 (3)	17 (14)	50 (42)	47 (40)	40 (34)	47 (40)	22 (19)	9 (8)
9 (33)b	0	1 (3)	4 (12)	28 (85)	12 (36)	11 (33)	10 (30)	0

Values are the number (%). Patients with gaps in followup are stratified by the years they were part of the cohort.

The year of followup was rounded to the closest integer, therefore a small number of patients who enrolled in early January 2007 and had a followup by mid August 2015 were included here.

The percentage of patients treated with a DMARD was high throughout all years of evaluation between 2007–2015, with a nonsignificant decline over the calendar years from 95% in 2007 to 87% in 2015, likely again due to a greater proportion of the sample with longer RA duration in later calendar years (e.g., with more opportunity to be off treatment) (Table 4 and the control chart shown in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). Stratification by year of enrollment showed similar rates of DMARD use in each strata (e.g., patients with 1 year of followup had similar rates of DMARD use across calendar years, shown on the diagonals of Table 4).

Table 4

CATCH patients treated with a DMARD (or biologic DMARD), stratified by year of enrollmenta

	2007	2008	2009	2010	2011	2012	2013	2014	2015	P
Numeratorb	86	342	614	842	1,015	1,143	1,240	1,250	1,078	–
Denominatorc	91	353	635	879	1,058	1,208	1,311	1,341	1,246	–
Proportion, %	95	97	97	96	96	95	95	93	87	0.34
Enrollmentd
2007	86 (95)	90 (99)	79 (94)	71 (93)	59 (89)	54 (90)	50 (96)	43 (93)	36 (88)	1.0
2008	–	252 (96)	257 (98)	233 (96)	199 (95)	177 (90)	163 (91)	141 (92)	105 (78)	0.65
2009	–	–	276 (96)	279 (97)	255 (95)	230 (92)	211 (91)	184 (91)	144 (84)	0.87
2010	–	–	–	259 (95)	266 (98)	241 (98)	210 (93)	189 (94)	164 (88)	0.91
2011	–	–	–	–	236 (97)	237 (98)	209 (95)	169 (88)	134 (79)	0.25
2012	–	–	–	–	–	204 (95)	210 (99)	184 (92)	142 (81)	0.33
2013	–	–	–	–	–	–	187 (99)	187 (99)	156 (93)	0.79
2014	–	–	–	–	–	–	–	153 (98)	151 (97)	0.91

Values are the number (%) unless indicated otherwise. CATCH = Canadian Early Arthritis Cohort; DMARD = disease‐modifying antirheumatic drug.

The numerator represents the total number of patients in CATCH treated with a DMARD or biologic DMARD in the cohort (for definitions of DMARD or biologic DMARD, see Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract).

The denominator represents the total number of patients in CATCH eligible for DMARD treatment in each calendar year, excluding patients who withdrew from followup during the calendar year.

Patients meeting the DMARD measure, stratified by year of enrollment.

The median time‐to‐DMARD start for all patients with an available date of diagnosis and a treatment start after diagnosis (n = 1,303) was 1 day with a mean ± SD of 34 ± 88 days (Table 5 and the control chart shown in Supplementary Appendix A, available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). When stratified by year of cohort entry, a trend toward decreasing time to starting a DMARD was seen (Table 5) (P = 0.02). Similarly, there was a nonsignificant trend toward an increased proportion of patients treated within the 14‐day benchmark, with 64% and 68% of patients who started DMARD treatment after diagnosis meeting this benchmark in 2007 and 2014, respectively. The percent treated within 14 days was higher at 79% when the 429 patients who received DMARD treatment prior to having a formal rheumatologist diagnosis of RA were included in this calculation.

Table 5

Time to disease‐modifying antirheumatic drug (DMARD) therapy, stratified by enrollment year in the Canadian Early Arthritis Cohorta

	Sample size, Rx ≥ Dxb	Time to DMARDb			% treated within 14 days
		Mean ± SD, days	Median, days	90th percentile	Rx ≥ Dxb	Include Rx < Dxc
Total	1,303	34.1 ± 87.6	1.0	93.0	66	75
Enrollment
2007	61	37.9 ± 93.2	1.0	100	64	76
2008	187	40.8 ± 98.2	1.0	121	63	74
2009	219	44.7 ± 122.4	1.0	111	66	74
2010	217	33.1 ± 85.2	2.0	83	65	72
2011	174	38.8 ± 91.0	1.0	123	67	76
2012	170	29.5 ± 68.9	1.0	88	66	73
2013	140	18.8 ± 35.3	1.0	66	71	78
2014	103	23.1 ± 51.2	0.0	69	68	79
P	–	0.02d	–	< 0.0001	1.0	–

Rx = treatment; Dx = diagnosis.

Limited to patients treated at or after diagnosis (Rx ≥ Dx).

Including patients treated before diagnosis (n = 429).

Statistically significant.

Discussion

The CATCH cohort is a multicenter cohort representative of early RA patients treated in settings of usual care in rheumatology practices across Canada. In this study, we observed high rates of adherence to the 3 performance measures that could be captured in CATCH, including high rates of yearly followup, DMARD use, and a high percentage of patients treated with a DMARD within the benchmark of 2 weeks. While overall rates of adherence to the measures were high, a small decline in yearly rates of rheumatologist visits and DMARD use was observed with increasing length of cohort followup. Conversely, a trend toward an increased rate of patients treated with DMARDs within the 14‐day benchmark was observed with increasing calendar year.

To our knowledge, this study is the first in which nationally endorsed system‐level performance measures 7 have been evaluated in Canadian patients. The use of real‐world data has helped inform measure development, as operationalization of the measures has identified a number of issues. In the following section, a rationale for updated specifications of some of the measures is discussed. A summary of the final recommended updates is shown in Supplementary Appendix A (available on the Arthritis Care & Research web site at http://onlinelibrary.wiley.com/doi/10.1002/acr.23439/abstract). First, when the measure capturing the percentage of patients in yearly followup was operationalized, both a fixed and a rolling time interval were planned to measure the percentage. Rolling intervals capture the time between patients’ 2 consecutive visits, resetting the clock so that the second visit is used as the new anchor time point for the subsequent followup visit and so on. This method, in theory, should provide a more reliable estimate of whether there were gaps in followup visits of >12 months between 2 consecutive visits. However, when trying to report on adherence to the measure by calendar year, using rolling intervals, it became apparent that a patient could have met and missed the measure within the same calendar year, so that reporting the adherence data by calendar year was impossible. Limitations also exist to using a fixed‐interval approach, as was done for this analysis. For example, a patient may have almost 24 months between 2 consecutive visits, but because the visits fall within 2 consecutive calendar years, the measure will be met in both years. To address this possibility, a secondary analysis was conducted to identify the number of gaps in care of >12 or >14 months duration between rheumatologist visits, which is a more sensitive indicator of gaps in care. Both methods are recommended for reporting this measure.

Additionally, although the percentage seen in the yearly followup measure was originally designed to be captured and reported on a yearly basis, a 14‐month window seemed more realistic. For example, a physician may request a yearly followup with a patient, but due to a scheduling preference this appointment gets booked in 13 or 14 months, such that the performance measure would not be met, yet this length of time may not truly reflect a gap in care. Additionally, in some Canadian provinces physicians may be remunerated more for followup visits billed at an interval greater than 1 year. This fact means there may be an incentive to booking patients just over 1 year, which may also not represent a true gap in care. Indeed, the adherence to the measure improved, and the number of gaps declined significantly, when using a 14‐month rather than a 12‐month window. Ideally, a 14‐month interval for assessing followup would be optimal, as in clinical practice insurers sometimes require 12 months between appointments, so that a 12‐month interval is impossible to meet. Thus, the measure results reported for a given calendar year require 12‐month look‐back and look‐forward periods if the measure is intended for high stakes applications, to provide the most accurate performance assessment. However, measuring adherence to 14‐month windows of followup on a yearly basis over time (e.g., >1 year) becomes problematic for reporting, as the windows of expected followups will eventually no longer fall neatly within calendar years for reporting. Additionally, further evaluation should be conducted to determine whether adherence to a 12‐month versus a 14‐month window of followup impacts any patient outcomes.

For the time‐to‐DMARD measure, over 400 patients were treated with a DMARD before their physician‐reported date of diagnosis. This treatment may have been due to a variety of reasons. For example, patients may not have fulfilled RA criteria or had undifferentiated or palindromic presentations and were treated with a DMARD before a formal diagnosis of RA was made. Also the paradigm for the diagnosis of patients recruited before the 2010 ACR/EULAR criteria 11 were published may have meant that patients were being treated off‐label before a formal RA diagnosis was made according to the older ACR criteria 10. When calculating the median and 90th percentile, as recommended when reporting on this measure 7, this calculation incurred negative values for the patients treated before diagnosis. Therefore, the performance measure was reported in 2 ways. The first included only those patients treated after the diagnosis of RA was made, as ensuring no delays is critical in patients referred without a diagnosis and without treatment, and this would represent an appropriate target for quality improvement if long wait times were observed. Secondly, the performance measure included all patients with an available diagnosis and treatment start date, and considering the 14‐day benchmark met when treatment is initiated before diagnosis. Reporting both of these metrics may be useful to understand practice patterns in health systems. A final consideration for this measure is the potential for gaming (e.g., physicians delaying a documented diagnosis of RA to decrease the time‐to‐DMARD start). While an unlikely consequence of measurement, such a practice is a possibility in high‐stakes measurement. Of note, physicians participating in CATCH were not aware they were being evaluated on measure performance.

The quality of care in rheumatology is not routinely measured in Canada outside of the context of local quality improvement projects and research using administrative data 13, 14, 15 or chart reviews 16, 17. In contrast, in the US, rheumatology quality measures are reported in national programs used for physician quality reporting tied to reimbursement. The ACR Rheumatology Clinical Registry 18 was used by rheumatologists for the Physician Quality Reporting System and demonstrates that registries are useful vehicles for quality monitoring and reporting. Recently, the ACR has developed the Rheumatology Informatics System for Effectiveness (RISE) registry 19. RISE passively extracts electronic medical records (EMR) data from participating practices for quality measurement and reporting. The ACR recently developed a set of electronic quality measures for use in RISE to capture access to treatment, as well as disease activity and functional status monitoring and tuberculosis screening prior to biologic or small molecule therapy 3, 4. The DMARD performance measure from the present set 7 was designed to be harmonized with the ACR DMARD measure, to ensure comparable measures between countries; however, no additional overlap exists between the measure sets.

Current data from RISE on adherence to the ACR's e‐measure for DMARD use revealed similarly high rates of DMARD use (91% adherence) in a denominator of over 57,000 RA patients 4. In contrast, significant gaps in DMARD use are still reported in population‐based studies 15, 20, 21, even among patients under the care of a rheumatologist 20. This difference could be because rheumatologists participating in CATCH and RISE may be more aware of quality measures and more likely to adhere to them than nonparticipating rheumatologists. The difference also may reflect an enrollment bias, whereby physicians may be less likely to enroll patients in cohorts who are not expected to follow up or comply with medication recommendations (although unlikely with RISE, as it is EMR‐based enrollment). Alternatively, the completeness of administrative data compared to cohort and EMR data may also play a role in the disparate results.

Delays in DMARD starts have been associated with worse outcomes for patients, including increased radiologic progression and need for surgical interventions and lower remission rates 22, 23, 24, 25. Most significant delays to therapy occur due to delays in access to rheumatology, and time‐to‐DMARD start once a patient is seen by a rheumatologist is often negligible 26, a finding confirmed by our study. In comparison, wait‐time benchmarks for DMARD start times are more stringent in our Canadian set than in the NICE quality standards from the UK, yet higher rates of adherence are reported in our study (75% within 14 days of diagnosis in CATCH compared to 53% treated within 6 weeks of referral in the UK) 27. Unfortunately, the complementary measure of waiting time to rheumatology care, that captures the time between rheumatologist referral and first visit for patients with RA 7, was not captured in CATCH and is being evaluated in other data sources, including triage databases across the country. In the UK, adherence to the NICE quality standard wait‐time benchmark of 3 weeks for rheumatologist consultation was poor (38%) 27, likely contributing to the lower rates of timely treatment seen, as the measure for time‐to‐treatment was anchored at the referral date.

Guidelines, quality standards, and quality indicators recommend that RA patients be seen at least yearly in followup to evaluate their disease status, screen for comorbidities, and review medications 5, 7, 28, 29. There has unfortunately been little published evaluation on RA patients who may have gaps in care or get lost to followup and on the consequences of lack of rheumatologist care. In a population‐based study from British Columbia, Canada, lack of continued rheumatology care was associated with lower rates of DMARD use (22% without any rheumatology care over the prior 5 years compared to 92% with continuous care) 14. While the present study is limited, as visit intervals were done per protocol, which may have biased the results, there was still a small decline in followup rates over time. This decline, however, may not be associated with a decline in care, as patients possibly withdrew or missed appointments with CATCH but continued seeing their rheumatologist. In addition, patients who missed followups may have been in low disease activity or remission, and missing care may not have been associated with poorer outcomes. Further study is warranted to evaluate the impact of gaps in care on patient DMARD use and most importantly on patient outcomes.

While this study represents the most comprehensive examination of the recently published system‐level performance measures for inflammatory arthritis in Canada, there are a number of limitations. While treatment is at the discretion of the treating rheumatologist, care received may not be entirely representative of usual care, due to selection bias from participating rheumatologists and patients. Physicians agreeing to participate in CATCH may be more motivated to treat patients according to guidelines and best practices. Also, through the research conducted, they may receive more practice feedback than the average rheumatologist in Canada, possibly leading to higher rates of adherence to the measures. Furthermore, patients who dropped out of CATCH or who had missing data possibly could have created a reporting bias, although numbers of dropouts were small after the first year (between 11 and 68 per year). Additionally, adherence to the performance measure capturing the percentage seen in yearly followup observed in this study may represent an overestimation compared to usual care, as followup is dictated by a protocol.

The original performance measures descriptions also included some exclusion criteria, which could not be easily operationalized in the context of CATCH. For example, for the DMARD measure, patients with a pregnancy during the measurement year should be excluded from measurement; however, this variable was inconsistently captured in the data and, given the small numbers, likely had minimal impact on results and was not formally applied in this analysis. Additionally, given the source of the data, DMARD use represents DMARDs prescribed rather than DMARDs taken, as information on patient nonadherence to DMARD initiation or treatment is not available. Also, only 3 of the performance measures could be captured in this data set. While 2 of the performance measures required different data sources (percentage of patients seen by a rheumatologist and rheumatologists per capita), the addition of referral dates to CATCH data capture would allow the waiting‐time measure to be reported. Lastly, while the measures were developed considering existing national and international guidelines for RA, they were developed for the Canadian context and may not be widely generalizable. Despite these limitations, this work provides a clear set of benchmarks for further evaluation of these measures in other data sources, as this cohort likely represents a best‐case scenario.

This study represents the first time the system‐level performance measures have been tested and highlights the fact that testing performance measures is crucial prior to implementation, due to potential issues in measure operationalization and/or interpretation using real‐world data. Further testing of the measures in different data sources is planned, including in administrative data, EMR data, and other clinical cohorts. Additionally, future studies examining patient and provider factors that impact measure performance will be important to conduct, to better understand how to improve any observed gaps in care. While the rates of adherence in this cohort were high, lower adherence will perhaps be observed with other data sources, but the current results may be used in benchmarking.

Author contributions

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Barber had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design

Barber, Lacaille, Marshall, Barnabe, Hazlewood, Thorne, Ahluwalia, Bykerk.

Acquisition of data

Bartlett, Boire, Haraoui, Hitchon, Keystone, Tin, Pope, Denning.

Analysis and interpretation of data

Barber, Schieir, Bykerk.

Role of the study sponsor

Companies providing financial support for the Canadian Early Arthritis Cohort had no role in the study design or in the collection, analysis, or interpretation of the data, the writing of the manuscript, or the decision to submit the manuscript for publication. Publication of this article was not contingent upon approval by any of the financial supporters.

Click here for additional data file.