Are interventions to reduce interruptions and errors during medication administration effective?: a systematic review.

BACKGROUND: Medication administration errors are frequent and lead to patient harm. Interruptions during medication administration have been implicated as a potential contributory factor.
OBJECTIVE: To assess evidence of the effectiveness of interventions aimed at reducing interruptions during medication administration on interruption and medication administration error rates.
METHODS: In September 2012 we searched MEDLINE, EMBASE, CINAHL, PsycINFO, Cochrane Effective Practice and Organisation of Care Group reviews, Google and Google Scholar, and hand searched references of included articles. Intervention studies reporting quantitative data based on direct observations of at least one outcome (interruptions, or medication administration errors) were included.
RESULTS: Ten studies, eight from North America and two from Europe, met the inclusion criteria. Five measured significant changes in interruption rates pre and post interventions. Four found a significant reduction and one an increase. Three studies measured changes in medication administration error rates and showed reductions, but all implemented multiple interventions beyond those targeted at reducing interruptions. No study used a controlled design pre and post. Definitions for key outcome indicators were reported in only four studies. Only one study reported κ scores for inter-rater reliability and none of the multi-ward studies accounted for clustering in their analyses.
CONCLUSIONS: There is weak evidence of the effectiveness of interventions to significantly reduce interruption rates and very limited evidence of their effectiveness to reduce medication administration errors. Policy makers should proceed with great caution in implementing such interventions until controlled trials confirm their value. Research is also required to better understand the complex relationship between interruptions and error to support intervention design.

Introduction

Medication administration errors (MAEs) have received relatively limited research attention despite evidence demonstrating that they are more likely to result in serious harm and death compared to other medication errors.1 2 MAEs have been found to account for 34% of preventable adverse drug events,3 and between 9% and 27% of medication doses administered contain at least one clinical MAE (eg, wrong dose, wrong time).4–8 Studies measuring procedural MAEs (eg, failure to check a patient's identification, use of aseptic technique) report error rates of between 74% and 83%.7 9 10 However comparisons between studies are greatly hampered by differences in error definitions applied.6 11 A review by the US Agency for Healthcare Research and Quality concluded that there is a particular lack of research evidence regarding factors associated with MAEs, significantly limiting the development of effective interventions.12

Qualitative studies and retrospective reviews of incident reports have increasingly suggested that interruptions are a contributory cause of MAEs in hospital.13–15 More recently a controlled direct observational study in two hospitals showed a significant association between interruptions to nurses during medication administration and the rate and severity of MAEs.7 These findings are consistent with laboratory studies in other settings which have shown that interruptions can negatively influence general task performance.16–20

In the aviation industry interruptions have been associated with major negative events21 and identified as a risk factor for task errors. As a result of growing concerns regarding the potential hazards of interruptions to work, in 1981 the Federal Aviation Authority in the US mandated, the ‘Sterile cockpit’ which has now become part of standard operating practice on most commercial airlines. In essence, the aim is to remove all forms of possible interruption to air crew while undertaking safety critical tasks such as taking off and landing a plane. This includes prohibiting non-essential conversations unrelated to the critical task.

Two decades after the introduction of the sterile cockpit, several anecdotal and descriptive accounts of hospitals applying aspects of the sterile cockpit concept began to appear.22–26 These included applying a ‘no interruption zone’ for preparing and checking medications and the use of ‘no interruption vests’ combined sometimes with staff and patient education and ward signage. The introduction of such interventions has at times been associated with considerable controversy.27 In 2011 in the UK, a hospital which introduced wearing of vests (printed with the words ‘Do not disturb’) by nurses administering medications to discourage interruptions, abandoned the programme following media reports which suggested the scheme ‘sent out the message that patients should not “bother” nurses’.28–30 Many nurses were alarmed that the intervention was abandoned as they believed it was critical to improving safety and the efficiency of care.28

A range of evaluation studies to assess the effects of ‘do not interrupt’ interventions has been undertaken. However, this evidence has not been brought together to allow policy makers and health care organisations to make informed decisions regarding the value of such interventions. We aimed to address this gap by conducting a systematic review of studies assessing the effectiveness of interventions aimed at reducing interruptions to nurses during medication administration on interruption and MAE rates.

Methods

Search strategy

The databases MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, CINAHL and PsycINFO were searched using a combination of subject headings and free text for relevant articles (see online supplementary appendix 1). The free text terms were limited to the title and abstract fields. The search terms used were related to nursing, medication, communication and interruptions, and intervention studies. The Cochrane Effective Practice and Organisation of Care Group reviews were also searched. The reference lists of the included studies were hand searched to identify further relevant studies. The search strategy was expanded by searching Google and Google Scholar using similar free text search terms to the database search strategy. All searches were carried out in September 2012.

Inclusion criteria and assessment

Studies reporting on the effects of an intervention or combination of interventions which included the aim of reducing interruptions or distractions to nurses during medication administration in any setting (eg, hospital, aged care facility) were included. Inclusion criteria required studies to report quantitative data, based on direct observation of interruptions or MAEs comparing intervention and non-intervention groups, or pre and post intervention. Studies that did not use direct observation, but relied only on other methods, such as self-report or qualitative assessment, for outcome measurement were excluded. Studies which used both direct observation and other methods were included. However only the details of the outcomes measured by direct observation were reported. All types of study designs were eligible for inclusion. Conference abstracts were excluded.

MZR conducted the search strategy. The titles and abstracts of all references identified in the database searches were assessed by two reviewers (JIW and MZR) to exclude those that clearly did not meet the inclusion criteria. The full-text articles of the remaining potentially relevant references were accessed and examined by the two reviewers for eligibility. Any discrepancies between the two reviewers were discussed to arrive at final agreement on the study's eligibility.

Data extraction

Data were extracted on the study design, location and setting, definitions, interventions implemented, outcomes measured and intervention effects. The location and study setting included the country, number of hospitals, number of wards and types of wards where the study was conducted. We examined and reported on the quality of the research designs and methods applied in terms of the application of clear definitions for ‘interruptions’, ‘distractions’ and ‘MAEs’; sample sizes; attempts to assess inter-rater reliability between observers; the use of significance testing to measure differences between comparison groups; and the independence of the data collectors from the study sites.

Further data extracted from studies included: interruption rates overall, interruption rates by source (ie, who or what was the source of the interruption), interruption rates by stage of medication administration, time taken to complete medication administration and MAE rates. Details of the sample sizes in relation to the: number of administered medications observed; number of patients administered medications; number of medication rounds; number of hours of observation and the number of nurses observed.

Results

A total of 626 citations were retrieved from the database and other searches. Eleven papers met the inclusion criteria (figure 1).22 26 31–39 Of these, two papers by Nguyen et al35 36 reported results from the same study with the second paper reporting data from a further follow-up period in addition to the original baseline data. Thus only results from the second paper are presented in the results and tables. Two papers by Kliger et al33 34 also presented findings on the same project, but with an extended sample and follow-up data, thus both were included as separate studies. Studies by Kliger et al and Nguyen et al36 were related to the same broader project, the ‘Integrated Nurse Leadership Program’. Online supplementary table S1 reports details of the studies, including setting, interventions implemented, outcomes and methods of measurement.

Figure 1

Results of search strategy and selection process to identify studies for review; *Two papers reported the same study; †This included 8 studies indentified through the database searches and the two studies identified through Google and Google Scholar searches.

A further seven studies, which were closely related to the review topic and investigated ‘Do not interrupt’ interventions, were excluded as they used only self-reported measures for the outcomes.40–46 Of these, three studies used a questionnaire self-administered by nurses after medication administration,40 42 45 two used self-administered general surveys about nurses’ perceptions of interruptions before and after the intervention,43 44 one relied only on the medication error incident reporting system41 and one had nurses record interruptions they experienced.46

Study characteristics

The majority of studies (n=7) were conducted in the USA,22 26 31–34 36 one in Canada39 and two in Europe.37 38 Studies were predominantly undertaken in one ward in one hospital (n=6, 55%). The type of ward studied varied from general medical wards to intensive care and chemotherapy day care. Nine studies (91%) used a before and after design with no control. The remaining study used a quasi-experimental three-group design.22 Three studies33 34 36 were related to the same project, the ‘Integrated Nurse Leadership Program’.

Online supplementary table S2 lists the study sample sizes and table 1 the outcomes measured in the included studies. Change in interruption rates overall was the most frequently measured outcome (n=9), followed by interruption rates by source (n=5, 50%).

Table 1

Outcomes measured by direct observation in included studies

Outcome	No. of studies (% of all studies)*
Interruption rates overall	9 (90)
Interruption rates by source	5 (50)
Interruption rates by medication administration stage	3 (30)
Time taken to complete medication administration	3 (30)
Medication administration error rates	3 (30)

*Some studies reported multiple outcome indicators.

Of the eight studies that used more than one observer, only one study estimated and reported the inter-rater reliability measure, Cohen's κ coefficient.38 Another study used percentage agreement between observers to estimate inter-rater reliability,22 while the remainder studies did not report calculation of inter-rater reliability measures. Two studies mentioned that data collection methods were validated, but did not report evidence to support this statement.32 36 Freeman et al32 reported simultaneous observations were conducted and then observers discussed the results.32

Definitions of ‘interruptions’, ‘distractions’ and ‘MAEs’

Only four studies (40%) provided a definition of interruption26 31 37 39 with one making a distinction between interruptions and distractions.37 Observed changes in MAE rates following intervention were reported in three studies.33 34 36 These studies reported that they observed the occurrence of medication administrations and then compared observations with medication charts to identify errors. Observations were conducted by nurses employed at the study hospitals. MAEs were defined in terms of categories of errors, including wrong dose, wrong technique, wrong route, unauthorised drugs, omissions and extra doses.33 34 36 The outcome reported in all three studies was the proportion of doses administered correctly. Two studies also reported changes in categories of errors.33 34 None of the studies stated whether omitted doses and/or doses given without an order were included in the denominator. Additionally, none of the studies assessed the clinical severity of the errors.

Types of interventions implemented

All studies implemented more than one type of intervention in order to reduce interruptions (see online supplementary table S1). Interventions included marked quiet zones for medication preparation; signage requesting nurses administering medications to not be interrupted; checklists with the medication administration process carried by nurses; and vests, sashes or lighted lanyards worn by nurses during medication administration to alert others not to disturb the nurse. Diversion strategies involved other staff attending to phone calls and non-emergency patient inquiries and requests, so that the nurses administering medications were not interrupted. One study allocated a specific room for medication preparation that was protected from external stimuli.38

Effect of interventions on interruption rates

Effect on interruption rates overall

Online supplementary table S3 shows the results from direct observational studies on changes in overall interruption rates following interventions. Eight studies which reported overall changes in interruption rates used a before and after study design, and one made comparisons between three groups (one control group and two intervention groups) to measure differences between the three groups. Four studies collected post intervention data more than 6 months after implementation,33 34 36 38 while three did so 1 month or less after implementation.26 32 37

Seven studies showed a reduction in interruption rate post-intervention. Of these, three showed a statistically significant reduction in the interruption rate26 34 37 and four31–33 36 did not evaluate the statistical significance of the observed change. One study showed a statistically significant increase in the interruption rate following implementation of the intervention.38 One study measured and compared the effect of two different sets of interventions on interruptions22 and found that a combination of three interventions, vests, checklists and diversion strategies, reduced interruption rates significantly more than an intervention using diversion strategies alone.

Effect on interruption rates by source

Five studies looked at change in interruption rate by source22 26 32 37 38 and two assessed the statistical significance of the change.37 38 Relihan et al37 showed a significant decrease in the average number of interruptions per medication round hour from staff nurses, conversation, missing medication, noise and other causes; but not from personnel, other patients, visitors, doctors and telephone calls. Tomietto et al38 found a statistically significant decrease in the number of interruptions due to unavailable medications or materials, patient requests, attending to other activities, and answering telephone calls; but not from searching for information, answering patient call bells, managing documentation and other sources. Additionally, they found a statistically significant increase in interruptions from other staff members, which was responsible for an overall increase in the interruption rate following intervention implementation. Three studies did not report the statistical significance of the observed changes in interruptions by source.22 26 32 Freeman et al32 showed a decrease in interruptions by patients, nurses, family and pagers after intervention implementation. Anthony et al26 showed a decrease in both interruptions initiated by others and those initiated by the observed nurse. Pape et al22 reported a decrease in the number of distractions for both sets of interventions from doctors, phone calls, visitors, wrong dose medication, emergencies, external talking, and loud noises. Only one of their interventions was successful in reducing distractions from other persons, other patients, and neither intervention reduced distractions from missing medication.

Effect on interruption rates by medication administration stages

Three studies measured the effect of interventions on interruption rates by different medication administration stages.32 38 39 One showed a slight decrease in interruptions during intravenous push medication administration, but did not measure the statistical significance of this change.32 Another study showed statistically significant decreases in the interruption rates during electronic drug verification tasks and paper drug verification tasks, but not during intravenous pump programming tasks.39 The third study38 simply compared the proportion of interruptions occurring during preparation, verification and administration phases of medication administration and found post-intervention a statistically significant increase in interruptions during the preparation phase, and a significant decrease during the verification phase, with little change in interruptions during the administration phase.38

Effect on time taken for medication administration

Two studies measured the effect of interventions on the time taken for medication administration.31 37 Both reported a decrease in the time taken for medication administration, but no study evaluated the statistical significance of these changes.

Effect on MAE rates

Table 2 shows the reported effect of interventions on MAE rates in the three studies that used direct observation methods to measure errors.33 34 36 All three related studies measured the long-term impact of the interventions and showed a reduction in MAE rates. Interventions in these studies included the targeting of interruptions during medication administration, along with other interventions aimed more generally at improved medication administration safety. Only two of these studies evaluated and showed that this reduction in errors was statistically significant.33 34

Table 2

Effect of interventions on proportion of medication doses administered with error reported by studies using direct observation methods*

First author (year)	Interventions	Pre-intervention % of doses with error	Post-intervention		Statistical significance
			Measurement timing†	% of doses with error
Kliger (2012)34	Various‡	16.6	Immediately post	3.6	p<0.01
			12 months	2.0	p<0.01
Nguyen (2010)36	Signs, diversion strategies	2.0	6 months	0	–
			12 months	0	–
Kliger (2009)33	Various‡	14.6	6 months	8.2	p<0.05
			18 months	4.2	p<0.05

*All three studies were related to the same project, the ‘Integrated Nurse Leadership Program’.

†Post-intervention measurement timing refers to how long after intervention implementation measurement occurred.

‡Multiple interventions were designed and implemented (only some of which targeted interruptions) on each ward individually and were not reported in detail in the publications.

Discussion

The 10 studies overall provide weak evidence of the effectiveness of interventions to significantly reduce interruption rates and very limited evidence of their effectiveness to reduce MAE rates. This is primarily due to the small number of studies, and a failure to utilise robust study designs, sample sizes and appropriate statistical analyses. Thus policy makers and practitioners should proceed with caution in investing in such interventions, and efforts are required to establish controlled trials of such interventions to ascertain their value before widespread adoption.

Reporting and analysis of methods and results was weak for many studies. The majority did not report sufficient detail on the observed sample size, did not conduct inter-rater reliability assessments, used staff from the same hospital or wards as observers to collect outcome data which had the potential to lead to bias, and a significant proportion of studies did not assess the statistical significance of intervention effects. Definitions of outcomes were not clear for many studies. While interruptions were the focus of all studies, only four provided a definition and these varied substantially. MAEs were also not clearly defined in the three studies that reported them, with no study reporting whether omitted doses, or doses administered but not ordered, were included in the denominator. Such differences in definition have been shown to affect estimates of MAE rates and are therefore important to specify.6 15 Generalisation of study findings is difficult, since the studies were conducted predominantly in the USA, usually in only one hospital, and in a variety of types of wards. The methodological inconsistencies in studies examining interruptions and MAEs have previously been highlighted6 11 14 15 and this review confirms that intervention studies suffer from similar deficits.

Only five of 10 studies measured significant changes in interruption rates pre and post interventions. Four found a significant reduction and one a significant increase. Only three of 10 studies measured the key outcome indicator of MAE rates, with two showing a significant reduction, and one showing a reduction but not evaluating its statistical significance. However, these three studies had significant flaws. None used a controlled design or controlled for clustering in their analysis. Clustering takes account of the likelihood that nurses’ behaviours on any ward are likely to be similar and should be accounted for in the analysis.47 It is probable that such analyses would impact upon the results reported. The lack of control groups in all of these studies prevents any assessment of the extent to which other factors may have produced or contributed to the observed change in interruption and MAE rates. This is particularly problematic in studies where the follow-up period was a year or longer after the baseline data collection. Patterns of nurses’ work change over time48 due to a range of factors such as changes in nurse ratios, policies and other organisational initiatives, all of which may have influenced the results.38

Importantly, the studies by Kliger33 34 involved mixed interventions, many of which were not specifically related to reducing interruptions, but were focussed on other ways of reducing MAEs. Thus which intervention/s contributed to the observed reduced MAE rate is unclear. The three studies which used direct observation to measure MAE rates pre and post intervention reported that they used the same observational method and data collection process, the California Nursing Outcomes Coalition process for observing medication rounds.49 However, none of these studies defined interruptions, thus how observers decided what should be counted as an interruption remains unknown to readers. For example, it is unclear whether interruptions included or excluded self-interruption by nurses. All three studies involved multiple observers yet inter-rater reliability was not assessed and observers were staff employed in the study hospitals who were associated with the study wards and interventions applied. Both these factors are a potential source of bias. For example, it is likely that those evaluating the interventions may have had a strong interest in demonstrating their utility as they were also involved in the design and implementation of the intervention. The validity of the extremely low MAE rates reported by Nugyen at both baseline (2% errors) and at post-intervention (0% errors) is questionable in the light of the study design issues identified. While the under-pinning theory for all the interventions featured was the assumed causal relationship between interruptions and MAEs, none of the studies reviewed provided compelling evidence to support this relationship.

It should be noted that the related studies by Nugyen et al36 and Kliger et al33 34 were a part of a larger project whose aim was to train and empower nurses at each of their sites to implement and evaluate quality improvement initiatives. Thus having observers located at each site and directly associated with the interventions was important in achieving their objective. However, as a result of this design these studies provide limited evidence to inform the question of this review.

For a comprehensive understanding of the effect of interventions on interruptions, it would be useful to know who or what the sources of interruptions were, and the phases of medication administration when interruptions occurred. However, few studies examined interruptions at this level of detail, and fewer still looked at the statistical significance of any changes post intervention. Additionally, it is important to note that not all interruptions are negative. Some are necessary and contribute to patient safety. This issue was alluded to in some studies. Conrad et al31 indicated that they sought to measure changes in ‘unnecessary’ interruptions and defined necessary interruptions as those that add value to patient care. Anthony et al26 reported that they excluded some interruptions, such as those between a nurse preceptor and their orientee, with the assumption that they were necessary interruptions, yet did not systematically identify which other types of interruptions were ‘necessary’. An unintended consequence of interventions may be a reduction in necessary interruptions. However neither study which raised this issue sought to determine whether the rate of ‘necessary’ interruptions changed post-intervention. Assessing such unintended consequences can be challenging. For example, as Anthony et al26 pointed out, an interruption may be deemed necessary to patient care by one party, but not another. This issue has been highlighted in studies of pager calls, where recipients may judge some pager messages as unnecessary interruptions, yet those who initiate messages view them as critical to patient care.50 In general, studies failed to specifically investigate any unintended consequences of their interventions. Tomietto et al38 reported concerns that patients may fail to raise necessary interruptions as a potential negative consequence. This is a topic which should be addressed in future studies given the growing literature on unintended consequences of large scale interventions.51–53

Though we conducted thorough searches, it is possible that there were studies that were not identified. Additionally, it is important to recognise the possible effect of publication bias, where studies with positive results are more likely to be published than studies showing no effect or negative results. We attempted to minimise publication bias by conducting Google and Google Scholar searches in order to identify studies published outside of peer-reviewed journals.

While the identified evidence base for interventions targeting interruptions to nurses is weak, interventions aimed at reducing interruptions should not be dismissed until appropriate controlled trials have been performed to assess their value. However, the current evidence base is not sufficient to warrant widespread adoption of such interventions. Future research requires at a minimum, controlled before and after studies and preferably controlled randomised or cluster randomised studies across multiple sites to provide evidence of intervention effectiveness. Clear definitions of MAEs and the categories of errors included within definitions are required. MAE rates should be presented with different denominators to allow comparisons across studies. Detailed information about interruptions, their source and nature are needed, and interruptions and MAEs should be obtained using validated direct observational methods.

In parallel with more robust studies of intervention effectiveness, attention should be directed to better understanding the complex relationship between interruptions and error in clinical practice. There is still much to learn about this relationship. There are only very few studies which have demonstrated a significant correlation between interruptions and MAE rates.7 Yet this is the foundational motivation for ‘do not interrupt’ interventions. Studies to understand the mechanisms by which interruptions impact performance and factors which may mediate their effects among different groups in different settings are required. Interruptions which may negatively impact upon care, as well as those interruptions crucial to delivering positive patient outcomes should be considered in these studies.13 50 54