A multifaceted strategy for implementation of the Ottawa ankle rules in two emergency departments.

PROBLEM: Despite widespread acceptance of the Ottawa ankle rules for assessment of acute ankle injuries, their application varies considerably.
DESIGN: Before and after study. BACKGROUND AND
SETTING: Emergency departments of a tertiary teaching hospital and a community hospital in Australia. KEY MEASURES FOR IMPROVEMENT: Documentation of the Ottawa ankle rules, proportion of patients referred for radiography, proportion of radiographs showing a fracture. STRATEGIES FOR CHANGE: Education, a problem specific radiography request form, reminders, audit and feedback, and using radiographers as "gatekeepers." EFFECTS OF CHANGE: Documentation of the Ottawa ankle rules improved from 57.5% to 94.7% at the tertiary hospital, and 51.6% to 80.8% at the community hospital (P<0.001 for both). The proportion of patients undergoing radiography fell from 95.8% to 87.2% at the tertiary hospital, and from 91.4% to 78.9% at the community hospital (P<0.001 for both). The proportion of radiographs showing a fracture increased from 20.4% to 27.1% at the tertiary hospital (P=0.069), and 15.2% to 27.2% (P=0.002) at the community hospital. The missed fracture rate increased from 0% to 2.9% at the tertiary hospital and from 0% to 1.6% at the community hospital compared with baseline (P=0.783 and P=0.747). LESSONS LEARNT: Assessment of case note documentation has limitations. Clinician groups seem to differ in their capacity and willingness to change their practice. A multifaceted change strategy including a problem specific radiography request form can improve the selection of patients for radiography.

Problem

A gap remains between what we know and what we practise.1 2 3 Blunt ankle trauma is a common presentation to an emergency department; traditionally, radiographs are ordered for virtually all such patients, and typically 85% of these examinations do not find a fracture.4 According to clinical research findings, radiography is not always needed to exclude an ankle or foot fracture.4 5 The Ottawa ankle rules (box 1) are a clinical decision tool that aids the efficient use of radiography in acute ankle injuries.5 A systematic review published in 2003 confirmed that the Ottawa ankle rules accurately exclude ankle and mid-foot fractures in patients with ankle injuries and can reduce the number of unnecessary radiographs by 30-40%.6

Bone tenderness at the posterior edge or tip of the lateral malleolus (distal 6 cm), or

Bone tenderness at the posterior edge or tip of the medial malleolus (distal 6 cm), or

Inability to bear weight both immediately and in the emergency department

Bone tenderness at the base of the fifth metatarsal, or

Bone tenderness over the navicular bone, or

Inability to bear weight both immediately and in the emergency department

The Ottawa ankle rules are well known and widely used internationally.7 In the emergency departments in this study, they were either not being used or their application varied considerably. We aimed to identify strategies to improve the uptake of the Ottawa ankle rules in the emergency departments, and to determine if these strategies were effective in improving selection of patients with ankle injuries for radiography.

Background and setting

This study began at a major tertiary referral hospital serviced by a public radiology provider. After 12 months, the implementation strategies developed at the tertiary hospital were transferred to a community hospital serviced by a private radiology contractor, to assess if they would work in another setting. The study continued for a total of 20 months at the tertiary hospital and six months at the community hospital. These timeframes were determined by our funding. The two sites were chosen because clinical champions facilitated access to their radiology and emergency departments for the project, and because they had similar throughput of ankle injuries.

The case mix of the emergency departments differed substantially; the adult tertiary hospital saw proportionally greater volumes of higher acuity patients than did the community hospital, which saw both adults and children with predominantly lower acuity presentations.

Only radiographs ordered from the emergency departments were included in this study. At both hospitals, radiography requests were made on paper forms.

The target group consisted of all clinicians working in the emergency departments who were able to order radiographs—that is, triage nurses who had completed accreditation to order radiographs, nurse practitioners, and all medical staff. Over the study period, the target group comprised 315 clinicians at the tertiary hospital and 62 at the community hospital.

Appropriate ethical approval was obtained from the Human Research Ethics Committees of each hospital involved in the study.

Strategy for change

The study was underpinned by concepts for changing clinical practice and the implementation of evidence based guidelines.8 9 10 11 12 In particular, we adopted key strategies outlined by Grol, which included a barriers analysis, engagement of the target group in design and implementation of a plan which linked the change strategies to barriers, and finally evaluation of the outcomes.10 13

Baseline audit

A baseline audit established the gap between evidence and practice and provided a starting point from which to measure the extent of change in practice. The audit included all consecutive eligible patients over a retrospective six month period who presented to the emergency departments with an ankle or mid-foot injury.

Barriers analysis

An initial analysis identified potential barriers to evidence being used in routine clinical practice. We completed process mapping of several ankle patients from triage to discharge or admission, along with key informant14 interviews of emergency department and radiology staff.

Barriers fell into three groups:

Individual clinician—Knowledge of the Ottawa ankle rules, concern about missing a fracture, lack of confidence in clinical ability to exclude a fracture without a radiograph, and lack of knowledge about which radiograph to order were key factors. Perceptions that “ankles are not important,” “the patient expects a radiograph,” and “it’s better to carry out radiography, otherwise the patient will re-present” were common.

Social context—Many staff felt obliged to order a radiograph if patients were unhappy with lengthy waits to be seen.

Organisational context—High staff turnover due to rotating staff, staff on shift work, no negative consequences of ordering unnecessary radiographs, and, at the tertiary hospital, triage nurses ordering radiograph without clinical examination at busy times to improve flow of patients within the emergency department were all important barriers.

Engaging the target group

At the tertiary hospital, the results of the baseline audit and barrier analysis were presented to an emergency department consultants’ meeting to gain support in principle. We identified change champions and clinical opinion leaders in both the emergency department and radiology, and convened a multidisciplinary steering group, which helped design the change strategies and drive the changes.

At the community hospital, we presented the results of the baseline audit and barrier analysis to a general staff meeting with the director and acting manager of the emergency department present, to provide support and endorsement.

Linking change strategies to barriers

The change plan was a multifaceted change strategy designed to deal with the identified barriers and included:

Education

A new problem specific radiography request form (fig 1)

New radiography request form

Reminders

Audit and feedback

Using radiographers as “gatekeepers” (tertiary hospital only).

Barriers at the individual clinician level and in the social context were met by education during in-service tutorials. Barriers within the organisational context were met by introducing a new request form incorporating the Ottawa ankle rules (fig 1), reminders via posters and lanyard cards, audit and feedback, and empowering the radiographers to reject the old request forms or any incomplete new request forms (“gatekeeping”). Dealing with the lack of negative consequences for ordering unnecessary radiographs was outside the scope of this study. At the tertiary hospital, it had been common practice for triage nurses to order ankle radiographs without clinical examination during busy times to streamline the flow of patients through the emergency department. During the study period, a process of fast tracking all low acuity patients was independently implemented to reduce the time to definitive assessment (fast tracking was already in place at the community hospital). This meant there was less need for triage nurses to order ankle radiographs without clinical examination and this barrier no longer needed to be tackled. Strategies were tailored to each hospital, and apart from audit and feedback, were all introduced simultaneously (table 1).

Table 1

Multifaceted implementation

	Tertiary hospital	Community hospital
Education
Tutorials were organised to:- Teach the Ottawa ankle rules- Improve examination skills with hands-on teaching by an emergency physician- Provide instructions on how to use the new request form- Discuss verbal and written “prescription” to give to patients for whom radiography is not indicated. If the patient insisted on radiography, staff were advised to respect the patient’s wishes	- Multiple tutorials were run over a two month period, separately for medical and nursing staff at their regular in-house training sessions- Over the next four months, tutorials were run for each medical staff rotation, but no extra sessions were run for nursing staff because of the stability of the workforce- After the initial six months, education about the rules was left to the discretion of the emergency department consultants responsible for training	- Multiple sessions were run over a one month period, both at scheduled staff meetings and opportunistically when the emergency department was quiet- Additional sessions were arranged on an “as needed” basis
New request form
- Design and implementation of a problem specific radiography request form incorporating the Ottawa ankle rules as a decision tree	- Feedback was sought regularly from all stakeholder groups, resulting in iterative improvements- Version 3 was used at the end of the study (pregnancy was removed from the form as few pregnant women present to the tertiary hospital; it does not have an obstetrics department)	- Version 2 (developed at the tertiary hospital) was used without modification
Reminders
- Posters about the rules were placed around the emergency department- Each member of staff was given a paper guideline and a lanyard card containing the rules- The posters, guideline, and lanyard card all looked identical, and colours matched the new request form to “brand” the study and provide an instantly recognisable product	- Information about the rules was updated on the intranet- Occasional small features about the study were placed in the emergency department newsletter	- No additional reminders
Audit and feedback
	- Feedback to emergency department staff on outcomes and practice change at a midpoint review	- Feedback to emergency department staff on outcomes and change in practice at a midpoint review
Radiographers as “gatekeepers”
- Radiographers were taught about the Ottawa ankle rules and how the new request form was to be used- Radiographers were empowered to reject the old request form	- Senior radiographers enlisted to help maintain and drive the change in their work groups- Multiple sessions run over a one month period at the start of the study to all available radiography staff- Communication book left in radiography work area to document any implementation problems or queries	- The “gatekeeping” role was not requested of the radiographers working for the private radiology contractor

Evaluating outcomes

Assessment of documentation of the Ottawa ankle rules on the request form and in the case notes was used as a marker of the use of the rules in clinical practice. The hospital’s radiology imaging system was searched to determine if imaging had occurred for each ankle presentation. If there was no record, case notes were reviewed to confirm that no imaging had been performed. Request forms were reviewed for all presentations throughout the study, and case notes were reviewed for all patients in the baseline audit. Because of frequent delays in availability, case notes were reviewed only in the implementation period for patients who had no imaging requested, or those without documentation of the Ottawa ankle rules criteria on the request form. Definition of acceptable documentation of Ottawa ankle rules criteria was defined a priori by consensus of a group of senior emergency department consultants at the tertiary hospital and was applied consistently during the study period at both sites.

We assessed the proportion of patients sent for radiography in each emergency department, and we calculated fracture rates as the proportion of radiographs showing a definite fracture. The missed fracture rate was calculated as the proportion of patients who had no radiograph initially but subsequently presented to a public hospital in the metropolitan area within seven days and had a fracture demonstrated on radiography. These presentations were captured through the metropolitan-wide public hospital electronic clinical information system.

For all measures, we compared the results from the baseline and implementation periods by using Fisher’s exact test. The P value for statistical significance was set at 5%.

Effects of change

During the study, 1561 patients with ankle injuries presented to the two hospitals: 215 during the baseline period and 813 in the 20 month implementation period at the tertiary centre, and 244 during the baseline period and 289 in the six month implementation period at the community hospital. Age and sex of the patients at each hospital was similar, with most injuries occurring in the 20-29 year old age group in both sexes.

Documentation of the Ottawa ankle rules

Table 2 shows that documentation of the Ottawa ankle rules increased significantly on both the request forms (tertiary hospital 42.2%, community hospital 31.2%; P<0.001 for both) and in the case notes (tertiary hospital 37.2%, community hospital 29.2%; P<0.001 for both).

Table 2

 Documentation of Ottawa ankle rules, fracture rate, and use of new request form. Values are numbers (percentages) unless otherwise indicated; varying denominators indicate availability of request forms and case notes

Key measures	Tertiary hospital					Community hospital
	Baseline period(n=215)	Implementation period(n=813)	% difference (95% CI)	P value		Baseline period(n=244)	Implementation period(n=289)	% difference (95% CI)	P value
Patients referred for radiograph	206/215 (95.8)	709/813 (87.2)	−8.6 (3.9 to 13.3)	<0.001		223/244 (91.4)	228/289 (78.9)	−12.5 (6.4 to 18.7)	<0.001
“Positive” documentation†:
Request form	84/205 (41.0)	588/707 (83.2)	42.2 (35.3 to 49.0)	<0.001		76/223 (34.1)	147/225 (65.3)	31.2 (22.0 to 40.5)	<0.001
Case notes	123/214 (57.5)	767/810 (94.7)	37.2 (32.1 to 42.3)	<0.001		126/244 (51.6)	231/286 (80.8)	29.2 (21.1 to 37.1)	<0.001
Radiograph	115/205 (56.1)	665/707 (94.1)	38.0 (32.5 to 43.4)	<0.001		123/223 (55.2)	186/225 (82.7)	27.5 (18.9 to 36.1)	<0.001
No radiograph	8/9 (88.9)	102/103 (99)	10.1*	>0.05		3/21 (14.3)	45/61 (73.8)	59.5*	<0.001
Fracture identified on radiograph	42/206 (20.4)	192/709 (27.1)	6.7 (−0.3 to 13.3)	0.069		34/223 (15.2)	62/228 (27.2)	12.0 (4.6 to 19.8).	0.002
Missed fractures	0/9 (0)	3/104 (2.9)	2.9	0.783		0/21 (0)	1/61 (1.6)	1.6	0.747
Referrals on new request form	NA	622/707 (88.0)	—	—		NA	93/225 (41.3)	—	—
Forms completed correctly	NA	533/622 (85.7)	—	—		NA	81/93 (87.1)	—	—

CI = confidence interval; NA = not applicable.

*95% confidence intervals could not be calculated because the number within each cell for Fisher’s exact test was less than 5.

†Complies with consensus criteria for adequate documentation of Ottawa ankle rules.

Proportion of patients referred for radiography

The proportion of patients referred for radiography fell by 8.6% at the tertiary teaching hospital (95.8% v 87.2%; P<0.001) and by 12.5% at the community hospital (91.4% v 78.9%; P<0.001) compared with the baseline audit (table 2).

Proportion of radiographs showing a fracture

The increase in the proportion of radiographs that showed a fracture was not significant at the tertiary hospital (increased by 6.7% from 20.4% to 27.1%) but was statistically significant at the community hospital (increased by 12.0% from 15.2% to 27.2%; P<0.01; table 2).

In the baseline period no patients re-presented with a missed fracture. During the implementation period, three of the 104 patients (2.9%) at the tertiary hospital and one of the 61 patients at the community hospital (1.6%) did not undergo radiography on initial presentation and subsequently re-presented with a missed fracture. This compared with a missed fracture rate of zero in the baseline group (table 2).

Compared with baseline, a greater proportion of patients with ankle injuries were seen during the implementation period by nurse practitioners at the tertiary hospital (30/215 v 252/813) and by resident medical officers at the community hospital (6/244 v 70/289), and a smaller proportion of patients were seen by triage nurses in both emergency departments (56/215 v 126/813 at the tertiary hospital, 48/244 v 8/289 at the community hospital).

The effect of the change strategies on radiography referrals also varied by clinician group at both sites. The greatest reduction in proportion of patients referred for radiography was in the nurse practitioner groups (30/30 v 208/252 (decreased by 17.5%) at the tertiary hospital; 11/11 v 19/28 (decreased by 32%) at the community hospital) and resident medical officers at the community hospital (6/6 v 55/70; decreased by 21%), but these reductions were not statistically significant.

The proportion of referrals made on the new request form was 88% at the tertiary hospital and 41% at the community hospital. The proportion of new request forms completed correctly was 85.7% and 87.1% at the tertiary and community hospital, respectively (table 2).

Discussion

The Ottawa ankle rules were well accepted, but it was difficult to stimulate enthusiasm about the need for change in practice because minor ankle injuries were not perceived as important or life threatening, and because ordering unnecessary radiographs had no negative consequences. Guidelines that recommend the elimination of an established behaviour (such as ordering radiographs) are more difficult to implement than guidelines that recommend adding a new behaviour.15

Despite this, the change strategies implemented in this study achieved a significant increase in the use of the Ottawa ankle rules at both the tertiary hospital (37.2%) and the community hospital (29.2%). This change was three to four times higher than in a systematic review of 235 guideline dissemination and implementation strategies, which showed a median 10% improvement.9

However, the fall in the number of unnecessary radiographs at the tertiary hospital was considerably less than the 30-40% predicted by Bachman et al.6 This may reflect the hospital’s higher fracture rate at baseline (20% v Bachman’s 15%). The reason for the greater effect of the change strategy on documentation than on reduction in radiography referrals is not entirely clear. It is noteworthy that the reduction in unnecessary ankle radiographs could reduce costs in a public hospital setting, while it could potentially reduce revenue for a private radiology provider.

The number of missed fractures at both sites was small and not statistically significant. A more robust measurement of missed fracture rates could be achieved with longer periods of data collection.

Change in practice

Clinicians

The greatest change in practice was seen with the nurse practitioners at both sites and the resident medical officers at the community hospital. Nurse practitioners became champions of the implementation of the Ottawa ankle rules, and among the junior medical staff uptake may have increased because the new request form gave a clear, evidence based framework for clinical assessment. The reason for lack of change in radiography referral patterns by triage nurses (non-nurse practitioners) is not entirely clear. The shift in radiography referrals away from the triage nurses could account for some of the outcomes, but not for change in practice in other clinician groups.

Clinicians initially expressed concern that patients expected radiography to be performed. During the implementation period, staff reported increased confidence in explaining to patients why radiography was not needed. Only nine of the 165 patients who did not undergo radiography re-presented to the same or different hospital for a second assessment.

Workplace culture can be an important facilitator or barrier for change. Cultural aspects related to leadership, teamwork, and readiness to change were not evaluated in this study. The early adopters and change champions were crucial to our success, and their support allowed smooth passage of new practices through hospital committees and line management: hospital executives, directors of emergency and radiology departments, and all tiers of medical, nursing, and radiology staff.

New radiography request form

The new request form was well adopted at the tertiary hospital (88% of referrals were made on the new form). The staff acknowledged that it served not only as a memory aid but also described the appropriate population in which the Ottawa ankle rules should be used. Given the large volume of staff and the small proportion of permanent staff in the tertiary hospital’s emergency department, the new request form worked well as a change strategy and was used successfully by both rotating and shift working staff. At both sites, the new form was used correctly in over 85% of radiography referrals.

The reduction in unnecessary ankle radiographs was greater at the community hospital, even though the problem specific radiography form was used in less than half of presentations (41%). This may be due to the smaller size and stability of the community emergency department workforce.

A key strategy for implementing the new request form at the tertiary hospital was assigning a “gatekeeper” role to the radiographers. The radiographers accepted and performed this role, but for some it was confrontational and challenging.

Limitations

The observational nature of this study limits interpretation of the data. In addition, the study period was confined to the duration of the lead author’s fellowship.

Assessing change in practice by assessing documentation of the Ottawa ankle rules in the case notes was hampered by inconsistencies in the quality of documentation. Although most clinicians will carefully document the results of an electrocardiogram, they seem to be less motivated to document examination findings for an injury that is not life threatening.

The rate of missed fractures may have been higher than reported, as patients who subsequently presented to their general practitioner may have been referred to community based private radiology contractors; this would not be captured by the hospitals’ electronic clinical information system. The differing lengths of implementation periods and external factors such as seasonality may also have influenced these findings.

It is impossible to disentangle the outcome effects of the separate components of the change strategy when all, apart from audit and feedback, were started concurrently. A cost benefit analysis would further evaluate this multifaceted change strategy. The effect of this change strategy on length of stay in the emergency department, change in practice in requesting of other radiographs, and patient satisfaction could also be considered.

Lessons learnt

Several key learning experiences arose from this study.

Using documentation as a surrogate measure for clinical practice has limitations and is both time consuming and prone to error, but it is often the only way of evaluating clinical practice.

Different groups of clinicians seem to differ in their capacity or willingness to change their practice. In this study, nurse practitioners in both emergency departments and resident medical officers at the community hospital showed the greatest uptake of the Ottawa ankle rules.

A multifaceted change strategy including education; a new, problem specific, radiography request form; reminders; and audit and feedback, and using radiographers as “gatekeepers,” can result in improved selection of patients for radiography.