When rheumatologists report that they agree with a guideline, does this mean that they practise the guideline in clinical practice? Results of the International Recommendation Implementation Study (IRIS).

INTRODUCTION: The European League Against Rheumatism (EULAR) recommendations for the management of rheumatoid arthritis (RA) and the treat-to-target (T2T) principles have been developed in order to improve the treatment outcome of patients with RA, and have received broad attention. It is not clear, though, whether these recommendations are indeed followed up in clinical practice.
OBJECTIVE: To investigate if rheumatologists that report to agree with existing guidelines indeed follow them up in clinical practice.
METHODS: The International Recommendation Implementation Study (IRIS) included 132 participating rheumatologists from 14 countries. Participating rheumatologists received a questionnaire measuring their awareness/commitment with the EULAR/T2T recommendations and followed a dedicated educational programme. Subsequently, they were asked to enrol 5-10 patients with new-onset RA in the online IRIS database and monitor disease activity and treatment for a period of 1-2 years. Four recommendations (3 from the EULAR recommendations and one from the T2T recommendations) were selected on the basis of testability, and analysed with regard to compliance by participating rheumatologists.
RESULTS: In total, 72 of the 132 participating rheumatologists contributed 378 patients to the database. Of these participants, 70 (98%) agreed upfront with the recommendation that disease-modifying antirheumatic drug (DMARD) therapy should be started as soon as possible after diagnosis in every patient; 69 (96%) of the rheumatologists agreed with the recommendation that methotrexate (MTX) should be part of the first treatment strategy. When measuring the actual performance, it was found that the recommendation on early DMARD start was met in 253 (67%) of the recorded patients, and the recommendation on MTX in 225 (60%) of the recorded patients. Of the participants, 60 (83%) agreed that composite measures should be recorded regularly, but only in 134(54%) of the patients were composite scores actually recorded in ≥50% of patient visits.
CONCLUSION: Reporting to be compliant with EULAR recommendations and T2T principles, even after dedicated education does not mean they actually comply with it in clinical practice.

Rheumatologists report following guidelines in daily practice.

Some studies showed successful implementation of guidelines in clinical practice.

In our study, we aim to investigate whether rheumatologists who report following guidelines really do this in clinical practice.

This study may increase awareness of and improve adherence to guidelines and possibly also enhance implementation of guidelines in clinical practice.

Introduction

Over the past decades, rheumatoid arthritis (RA) guidelines and recommendations have been formulated with the aim of improving the quality of care.1–5 In rheumatology, international recommendations for the management of patients with RA include guidance about treatment decisions, such as the choice of initial and subsequent therapy, about monitoring disease activity as a measure of treatment success and about treatment adjustments. In addition, they emphasise the principles of shared decision-making, patient education, how to deal with comorbidities and the role of specialised nurses in the treatment care of RA.

These RA recommendations include concepts of ‘treating-to-target’ (further referred to as T2T) and ‘tight control’. The T2T approach requires that antirheumatic therapy in a patient should be chosen and adjusted in such a manner that clinical remission or low disease activity is achieved. The ‘tight control’ concept requires frequent assessments of disease activity in order to check the treatment goals and to avoid delays in optimal treatment. It is recommended that monitoring disease activity should be done by composite measures (DAS, DAS28, SDAI and CDAI).6–8

Surveys among rheumatologists have suggested that rheumatologists follow the recommendations for RA in clinical practice.9–11 Other studies, however, have suggested that recommendations are hardly practised outside clinical trials.12 13 These studies indicate that there is a discrepancy between reporting agreement with recommendations and the actual performance in clinical practice (implementation). Only a few studies have shown a successful implementation of recommendations, such as T2T and DAS steered therapy in clinical practice.14 15 Among the obstacles that may hinder successful implementation of recommendations are lack of awareness and lack of agreement,16–18 as well as the absence of proper treatment protocols.15 A previous study has suggested that dedicated educational programmes may help to implement clinical guidelines in practice.19

To test if such an educational implementation initiative may improve practice performance, the International Recommendation Implementation Study (IRIS) has been initiated. The first step of this study was to investigate if rheumatologists that report complying with certain European League Against Rheumatism (EULAR) recommendations on the management of RA and/or the T2T recommendations for RA actually do so when measured in clinical practice. In order to increase the likelihood of success, the participants had been offered a dedicated web-delivered educational programme about the recommendations before the start of the study.

Methods

IRIS is an implementation study with a 2-year follow-up. From December 2011, rheumatologists worldwide were approached via their national societies to participate in the study. One hundred and thirty two rheumatologists from the following 14 countries (Bosnia, Brazil, Croatia, Cyprus, Greece, Italy, Malta, the Netherlands, Nigeria, Poland, Portugal, Russia, Spain and Turkey) agreed to participate.

Participants received a questionnaire with questions about the awareness of: (1) agreement with (2) and adherence to (3) the EULAR and T2T recommendations formulated in 2010 (see online supplementary attachments I and II).6 8 Four recommendations were preselected by us for further testing, based on their appropriateness for measuring awareness (1) and agreement (2), but in addition also on the appropriateness of actually measuring performance in clinical practice (3) (database).

The selected recommendations were:

‘Treatment with synthetic disease-modifying antirheumatic drugs (DMARDs) should be started as soon as the diagnosis of RA is made’

‘Methotrexate (MTX) is part of the first treatment strategy in patients with active RA’

‘When MTX contraindications (or intolerance) are present, the following DMARDs should be used: leflunomide, sulfasalazine or injectable gold’

‘Measures of disease activity must be obtained and documented regularly as frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission’

Participating rheumatologists took part in an educational programme, that included (1) the appraisal of the ‘EULAR recommendations for the management of RA’ article and the ‘T2T recommendations’ article; and (2) watching an online video in which the recommendations and the aims of IRIS were further explained and elucidated by expert rheumatologists and researchers. In addition, participants could follow an online training programme about the Measurement of efficacy of Treatment in the Era of Outcome of Rheumatology (METEOR) registration tool.

Next, participants were required to enrol 5–10 patients with newly diagnosed RA, and to record disease activity assessments and treatment adjustments as often as they thought this would be appropriate. Treatment choice and monitoring frequency was entirely at the discretion of the participating rheumatologist. Registration was performed in the METEOR database,20 21 a combination of a database and an online tool to register data from patients with RA and monitor them in daily practice. The patients were followed for 1–2 years. During follow-up, participating rheumatologists received one of the EULAR and/or T2T recommendations per email every week as a reminder to encourage them to comply with the recommendations in clinical practice.

The first participant started with the educational programme in March 2012 and the final participant started in February 2014. The database is still ongoing and will close 2 years after the last participant has added the last patient (February 2016). The participants received a payment of €250 per included patient to compensate for the work in this study.

Analysis

The primary analysis was a comparison between the rate of agreement with each of the four selected recommendations and the rate of actual compliance with them when measured in the METEOR database. The recommendations were operationalised as follows:

EULAR recommendation 1: ‘Treatment with synthetic DMARDs should be started as soon as the diagnosis of RA is made’; was operationalised as: The proportion of patients in whom the time interval between the date of diagnosis and the date of start of a DMARD (disease modifying anti-rheumatic drug) was ≤4 weeks.

EULAR recommendation 2: ‘MTX is part of the first treatment strategy in patients with active RA’; was operationalised as: The proportion of RA patients in whom MTX was (part of) the first treatment strategy.

EULAR recommendation 3: ‘When MTX contraindications (or intolerance) are present, the following DMARDs should be used: Leflunomide, sulfasalazine or injectable gold’; was operationalised as: The proportion of patients who did not start with MTX in whom leflunomide, sulfasalazine or injectable gold was prescribed as an alternative.

Treat to target recommendation 5: ‘Measures of disease activity must be obtained and documented regularly as frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission’. Three categories were formed: (1) ‘T2T always’ represents the number of patients for whom a composite score (DAS, DAS28, CDAI or SDAI) was reported at least every 2 months if moderate- or high disease activity (DAS>2.4, DAS28>3.6 CDAI>10, SDAI>11) was present; and at least every 7 months when low disease activity was measured in 100% of the visits; (2) ‘T2T sometimes/never’ when ‘T2T always’ was not met; and (3). ‘not reported’ when composite scores were missing in all of the visits.

Results

Of the 132 participating rheumatologists who agreed to participate in IRIS, 122 (92%) completed the questionnaire on awareness and agreement, took part in the web-based educational programme, and agreed to start in the patient enrolment programme.

During the follow-up period, 72 (55%) of the participating rheumatologists recorded 1.155 visits of 378 newly diagnosed patients in the database prospectively. The other participants declined participation (n=44) or were lost to contact (n=6) before including any patient in the METEOR database (figure 1). Reported reasons for declining were ‘lack of time’ and ‘consent withdrawn’. The remaining 72 rheumatologists were from eight countries (Bosnia, Cyprus, Greece, Italy, the Netherlands, Nigeria, Russia and Spain). We compared the results of the questionnaire from 72 rheumatologists who entered patients in the database with the 50 rheumatologists who did not, and found that agreement was similar between the two groups (see online supplementary attachments I and II).

Figure 1

Flow-chart.

Baseline characteristics of the participating patients were described in online supplementary attachment III table 1. Eight two per cent (n=300) of the patients were women. Mean (SD) age was 55 years and mean DAS was 3.1. Median time from diagnosis until first visit was 8 weeks.

Table 1

Comparison between reporting to follow the EULAR and T2T recommendations and managing patients in clinical practice

	Rheumatologists’ opinion about adherence (measured in 72 rheumatologists)*			Rheumatologists’ performance in daily practice (measured in 378 patients)†
	Always followed, n (%)	(Some) times/never followed, n (%)	Missing, n (%)	Always applied, n (%)	(Some) times/never applied, n (%)	Not reported, n (%)
EU 1. ‘Treatment with synthetic DMARDs should be started as soon as the diagnosis of RA is made’.	70 (98)	1 (1)	1 (1)	253 (67)	65 (17)	60 (16)
EU 3. ‘MTX is part of the first treatment strategy in patients with active RA’.	69 (96)	2 (3)	1 (1)	225 (60)	93 (24)	60 (16)
EU 4. ‘When MTX contraindications (or intolerance) are present, the following DMARDs should be used: leflunomide, sulfasalazine of injectable gold’.	59 (82)	12 (17)	1 (1)	15 (19)	78 (81)
T2T ‘Measures of disease activity must be obtained and documented regularly’‡	60 (83)	10 (14)	2 (3)	68 (27)	125 (51)23 in ≥75%;45 in ≥50%;27 in <50%;30 in none of the visits	54 (22)

*Always=rheumatologists report following this recommendation, sometimes/never=rheumatologists report following this recommendation sometimes or not, missing=no answer was filled in.

†Always=rheumatologists follow this recommendation, sometimes/never=rheumatologists follow this recommendation sometimes or not. Not reported=no information present on whether the recommendation is followed by the rheumatologist.

‡As frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission.

EULAR, European League Against Rheumatism; RA, rheumatoid arthritis; T2T, treat to target.

1. ‘Early treatment start’

Of the 72 participating rheumatologists, 70 (98%) had reported complying with the recommendation ‘Treatment with synthetic DMARDs should be started as soon as the diagnosis of RA is made’ (table 1). In 253 of the 378 (67%) patients who had been recorded in the database, a synthetic DMARD had indeed been prescribed within 4 weeks after the diagnosis.

In 65 of the 378 patients (17%), a DMARD was not started within 4 weeks. 55 of them received a DMARD in a later period. The median (IQR) duration until these patients received a DMARD was 15 (5–81) weeks (table 2).

Table 2

Average time from diagnosis (weeks) until first DMARD in patients in whom a DMARD was NOT started within 4 weeks after diagnosis

	Patients (n=65)(n, %)	Average time to start per therapy (median, IQR)
Methotrexate	41 (82)	13 (7–57)
Hydroxychloroquine	6 (12)	19 (12–606)
Sulfasalazine	1 (2)	10
Leflunomide	2 (4)	16 resp 189 weeks
Other drug	5 (23)	82 (25–336)
No therapy started	10 (15)	–
All therapies	55 (85)	15 (5–81)

For 60 of the 378 patients (16%), essential information was missing (in these cases the date of diagnosis or the start date of the first DMARD was missing).

2. ‘Start with MTX’

Of the 72 participating rheumatologists, 69 (96%) had reported complying with the recommendation ‘MTX is part of the first treatment strategy in patients with active RA’. In 225 of the 378 patients (60%), MTX has indeed been prescribed as (part of) the first treatment.

3. ‘Start with alternatives for MTX’

Of the 93 patients (26%) who did not start with MTX,15 (19%) received leflunomide, sulfasalazine or injectable gold as first treatment. However, 78 patients (81%) started with other medications than these three preferred alternatives. Six of these patients started with biologicals (table 3).

Table 3

Medication prescribed as initial treatment and average time in weeks from diagnosis until start in patients in whom MTX, leflunomide, sulfasalazine or injectable gold was NOT started as first DMARD

	Patients (n total=78)(N, %)	Average time to start per therapy (median, IQR)
Hydroxychloroquine (±NSAID)	12 (15)	0 (0–0)
Parental corticosteroid (±HCQ)	28 (36)	0 (0–0)
Oral corticosteroid (±HCQ)	28 (36)	0 (0–3)
NSAID/analgetics	3 (4)	0 (0–0)
Ciclosporin	1 (1)	0 (0–0)
Biological DMARD	6 (8)	64 (4–245)

DMARD, disease-modifying antirheumatic drug; HCQ, hydroxychloroquine; NSAID, non-steroidal anti-inflammatory drug.

4. ‘Frequent monitoring’

Of the participating 72 rheumatologists, 60 (83%) had reported compliance with the recommendation: ‘Measures of disease activity must be obtained and documented regularly as frequently as monthly for patients with high/moderate disease activity or less frequently (such as every 3–6 months) for patients in sustained low disease activity or remission’.

Of the 378 patients, 131 had less than two visits available in the database. For those patients with more than one visit recorded (247), we checked whether patients had been monitored according to this recommendation. Of these 247 patients, only 68 (27%) had been monitored in full compliance with this recommendation (‘T2T always’). Of the remaining patients, 23 (9%) had been monitored in partial compliance with this recommendation (75–100% of the visits); 45 (18%) had been monitored insufficiently (in 50–75% of the visits); 27 (11%) had been poorly monitored (<50% of the visits). Finally, 30 (13%) had not been monitored at all. All these remaining patients were categorised in table 1 under the heading ‘Sometimes/never applied’.

Discussion

The main conclusion of this study among rheumatologists practising in different parts of the world is that reporting compliance with EULAR/T2T recommendations does not necessarily mean that these recommendations are actually applied in daily clinical practice. We have found discrepancies between what rheumatologists report doing versus how they actually treat and follow patients in clinical practice.

Less than 60% of the recruited rheumatologists who agreed to participate, expectedly the more dedicated rheumatologists, finished the educational programme and included patients in the METEOR database, which is rather disappointing. Yet, even after participating in a dedicated educational programme, in which the participants were actively stimulated to follow their patients according to the recommendations, a strategy that has proven positive effects on implementation in many previous studies,19 22–24 rheumatologists still seem to be reluctant to practise that recommendation in real life and to register their performance. In fact, it is even worse: They report that they comply with recommendations but act differently in clinical practice.

What could explain this discrepancy? First, trivial logistic explanations may account. For instance, a patient may not show up for a visit, which in turn could lead to missing disease activity data within the recommended time period. Furthermore, it may need time to fill the gap between obtaining knowledge about the need for recommendations and actually practising them. Rheumatologists may agree with recommendations and be convinced by educational programmes, but still need more time than 1–2 years to actually change their practice. There is some evidence for this statement: In the study by Forsetlund et al,25 physicians who were actively stimulated to treat patients according to evidence-based practice were compared with physicians who only received access to evidence-based libraries, but no significant differences between the group in behaviour of decision-making was found. Their follow-up was 1.5 years, which was argued to be too short to change decision-making among physicians. A Dutch study also showed similar discrepancies between compliance with and actual application of recommendations about mental health in practice. In this study, no educational programme was used to encourage physicians to follow the recommendations.26

A more technical explanation for not following the recommendations in clinical practice is that we had to base our verdict about whether a recommendation was followed or not on the registration of rheumatologists' actions in the METEOR database. It is theoretically possible that recommendations were followed better than were actually recorded. However, all rheumatologists were explicitly informed about the procedures, as well as instructed to register their performance in the METEOR database and offered a training programme to optimally use that database. In addition, since we are not sure whether the participating rheumatologists have truly completed the offered educational programme, and we are not informed about the performance of rheumatologists without following the education, we cannot conclude from this analysis that the programme has influenced the behaviour of the rheumatologists. We did send out monthly emails with a recommendation in order to remind the rheumatologist on the project, which has been shown to be an effective tool in previous implementation studies,27 and may also have influenced performance. We have offered the educational programme via the internet, and a more effective approach could have been telephone interviews or meetings with physicians. Reviews have suggested that multifaceted strategies, such as educational meetings, educational resources and support from colleagues, are more successful to assist in implementation of recommendations,27–30 although another review has suggested that it is not clear yet which implementation strategies are best.31

An intrinsic reluctance to record daily practice in a database, although being user-friendly,20 may explain why only 72 (55%) of the rheumatologists who followed the educational programme succeeded in including patients. When we compared the rates of agreement with the recommendations between the rheumatologists who dropped out after the educational programme (n=50) with the 72 participants who effectively included patients, the responses were similar, thus excluding spurious selections of participants as an explanation for disappointing results.

Finally, technical data entry problems with the METEOR database may also have contributed to incompleteness of data. In 16–22% of patient data, information about the four selected recommendations was not reported. It is unclear if these patients are randomly missing, and if entered data are inconsistent. In addition, we can only speculate about the reasons for not complying with recommendations, as additional data on treatment steps, contraindications for medication, side effects and comorbidities are not recorded.

This study has strengths and limitations. An important strength of this study is that we have recruited study participants from all over the world, which increases generalisability. There are also limitations to this study. First of all, we have investigated the agreement and adherence to the EULAR recommendations of 2010, which were new at the time of study initiation but have been updated since then (online publication in October 2013, while inclusion and instruction in the current study was finished in March 2013). However, the updated recommendations did not differ much with respect to the four recommendations studied. The only difference that is relevant for the interpretation of this study is that injectable gold is not recommended anymore when MTX is contraindicated.

Another limitation is that we miss information about characteristics of the participating rheumatologists, due to privacy reasons. While rheumatologists from all over the world have participated in the study, it is still uncertain whether the study is fully generalisable to all rheumatologists.

In conclusion, the results of this study show that there is a discrepancy between reporting agreement with well-known and broadly accepted recommendations on treating-to-target, timing and choice of initial treatment as well as tight control in the management of patients with RA on the one hand, and the actual performance in clinical practice when measured on the other. A dedicated internet-based educational programme is most likely insufficient to change the attitude of the rheumatologists in this regard, at least in the short term. This observation has implications for the broadly advocated recommendation to implement quality-control initiatives in order to make practice performance more transparent: It looks as if the development and publication of evidence-based and consensus-based treatment recommendations do not suffice to change practice performance in rheumatology. Since these recommendations are usually a trade-off between best evidence and cost-effectiveness, it can be argued if nowadays patients with RA are indeed optimally treated in clinical practice. Further studies should focus on factors that explain the reluctance of rheumatologists to follow evidence-based treatment recommendations, in particular reluctance of MTX prescription in patients with comorbidities, and on strategies to overcome this reluctance. In addition, future studies should focus on investigating what type of education is most effective for implementing guidelines in clinical practice.