Measuring the relationship between interruptions, multitasking and prescribing errors in an emergency department: a study protocol.

INTRODUCTION: Interruptions and multitasking are frequent in clinical settings, and have been shown in the cognitive psychology literature to affect performance, increasing the risk of error. However, comparatively less is known about their impact on errors in clinical work. This study will assess the relationship between prescribing errors, interruptions and multitasking in an emergency department (ED) using direct observations and chart review. METHODS AND ANALYSIS: The study will be conducted in an ED of a 440-bed teaching hospital in Sydney, Australia. Doctors will be shadowed at proximity by observers for 2 h time intervals while they are working on day shift (between 0800 and 1800). Time stamped data on tasks, interruptions and multitasking will be recorded on a handheld computer using the validated Work Observation Method by Activity Timing (WOMBAT) tool. The prompts leading to interruptions and multitasking will also be recorded. When doctors prescribe medication, type of chart and chart sections written on, along with the patient's medical record number (MRN) will be recorded. A clinical pharmacist will access patient records and assess the medication orders for prescribing errors. The prescribing error rate will be calculated per prescribing task and is defined as the number of errors divided by the number of medication orders written during the prescribing task. The association between prescribing error rates, and rates of prompts, interruptions and multitasking will be assessed using statistical modelling. ETHICS AND DISSEMINATION: Ethics approval has been obtained from the hospital research ethics committee. Eligible doctors will be provided with written information sheets and written consent will be obtained if they agree to participate. Doctor details and MRNs will be kept separate from the data on prescribing errors, and will not appear in the final data set for analysis. Study results will be disseminated in publications and feedback to the ED. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This will be the first study to provide empirical evidence on the relationship between interruptions, multitasking and prescribing errors in situ.

The study will use validated direct observation work measurement methods and prescribing error measurement to generate evidence.

Although cognitive failures are similar across settings, the study will be carried out in one hospital potentially limiting the generalisability of the rates of errors to other settings.

Introduction

Since the publication of the Institute of Medicine's report To Err Is Human in 1999, there has been growing concern about the potential for medical errors due to the disruptive nature of clinical work environments.1 This has led to the implementation of a variety of interventions to reduce interruptions during clinical work.2–4 The negative consequences of interruptions have been demonstrated in experimental studies in the psychology literature.5–8 Similarly, multitasking has been shown to have a negative effect on task performance in experimental studies,9–12 notably during driving.13–15 However, while there are numerous descriptive studies detailing the frequency of interruptions and multitasking by health professionals in healthcare settings,16–24 few studies have assessed the impact of interruptions or multitasking on clinical outcomes or errors.

One study, using direct observation of nurses during medication administration across six wards in two Australian hospitals, found that each interruption was associated with 12.1% and 12.7% increased odds of procedural and clinical medication administration errors.25 Other studies have demonstrated interruptions that can result in changes in workflow and non-resumption of tasks.19 26–28 Frequent interruptions and multitasking have also been found to be associated with increased self-reported strain in hospital doctors.21 29

Prescribing errors are one indicator of clinical performance. A systematic review of prescribing errors in hospitals reported that they affect 7% of medication orders, 2% of patient days and 50% of hospital admissions.30 Prescribing errors also occur in emergency departments (EDs), and have been reported at a rate of 21.5% and 12.5% per medication order for adult and paediatric patients, respectively.31 32 Prescribing represents a discrete task during which interruptions or multitasking behaviours can be directly observed. Since EDs are high-pressure environments, where doctors have been shown to be interrupted more frequently than ward doctors,20 26 they present an ideal setting for observing interruptions and multitasking during prescribing. In the context of a lack of evidence about the association of interruption and multitasking with errors in clinical practice, this study aims to assess the relationship between interruptions, multitasking and prescribing errors in an ED.

Methods and analysis

Study setting and participants

The study will be carried out in the acute section of an ED in a 440-bed teaching hospital in Sydney, Australia.33 The study ED operates 24 h a day and in 2014 treated approximately 50 000 adult patients. The ED is organised in separate sections or ‘models of care’ in which patients of varying severity are treated. The acute section of the ED sees the most severe presentations, and includes three resuscitation bays and 12 beds. The resuscitation bays and beds are placed at the perimeter of the acute section, with a raised platform with computer terminals, termed ‘the bridge’, at the centre of the department. The bridge is where the majority of documentation and department management takes place.

All fully qualified ED doctors will be invited to participate in the study. A qualified doctor in Australia has completed a medical degree and 1 year of on the job training. This includes resident medical officers (RMO; 1–2 postgraduate years’ experience), senior resident medical officers (SRMO; 2–3 postgraduate years’ experience), registrars (at least 3 postgraduate years’ experience and completion of relevant training) and staff specialists (at least 5 postgraduate years’ experience and completion of relevant training). Written consent will be obtained from doctors who agree to participate. Interns, who have completed their university education but who have not yet obtained full medical registration, will not be invited as they may be less comfortable with the presence of an observer.

All qualified doctors listed above can prescribe medications. In Australian EDs, doctors prescribe medications that need to be administered in the ED, as well as on wards when a patient is admitted to the hospital. In the acute section of the study ED, approximately 60–70% of patients are admitted to the hospital.

Direct observations

Doctors who consent to participate in the study will be observed for 2 h time intervals while they are working on day shift (beginning at 08:00 and ending at 18:00). Observers will shadow doctors at proximity in all areas of the ED, excluding the resuscitation bays, as required by the ethics study approval. Approximately equal numbers of doctors by seniority will be followed, and observation sessions will be spaced so that any time between 08:00 and 18:00 is covered by roughly the same number of sessions so as to be representative of ED work during the full time span of the day shift. An observational sampling matrix has been prepared to support this process.

Sophisticated time-and-motion methods will be applied to document doctors’ work using the Work Observation Method by Activity Timing (WOMBAT) system.34 WOMBAT provides data collection software which allows observers to record tasks undertaken during doctors’ work using a handheld tablet computer. All tasks entered are automatically time-stamped. The task categories and their definitions to be used in this study are shown in table 1, and have been adapted to this study setting based on other similar studies.23 26 34 35 For each task, the person with whom it is performed (staff specialist, registrar, resident or intern, nurse, patient, relative, ambulance officer, pharmacist, allied health, administrative personnel, security personnel or police, other, no one), the location, communication tools used (mobile phone, landline phone, computer, paper) and any interruptions or multitasking will be recorded.

Table 1

Dimensions, categories and subcategories to be used for data collection in the WOMBAT system

Task	Definition
Direct care	An activity directly related to the care of one patient for example, examining a patient, performing a procedure, communication with a patient or relative
Indirect care	An activity indirectly related to patient care for example, ordering tests, reading documents, washing hands, gathering and returning equipment
Documentation	Recording of patient-related information on paper or computer for example, writing patient notes
Clinical communication	Any discussion on clinical issues, excluding communication with patients/relatives for example, planning care with another health professional, requesting medical or nursing consultation, handover
Management communication	Discussion related to running of the department and other administrative issues for example, rosters, bed allocations, employment issues
Social communication	Conversations unrelated to work for example, personal phone calls
Prescribing	Writing one or more medication orders on a medication chart, form or prescription pad
In transit	Moving between areas of the department
On break	Breaks from work for example, eating lunch

WOMBAT, Work Observation Method by Activity Timing.

When prescribing is observed, this will be recorded in the WOMBAT system as a prescribing task (table 1). In the ED, medication orders are handwritten on the following types of paper-based charts: (1) the Australian national inpatient medication chart (NIMC),36 (2) fluid form, (3) heparin infusion chart, (4) insulin chart and (5) personally controlled analgesia form. For the NIMC, the chart sections include once only, when required, variable dose, venous thromboembolism (VTE) prophylaxis, warfarin, regular page one and regular page two. Medication orders written during observed prescribing tasks will be assessed for errors after the observational session. To facilitate this, details of which medication chart, sections and lines were completed during the observed prescribing task will be recorded in WOMBAT, and the patient medical record number (MRN) will be noted on a separate sheet. The MRNs for each subsequent prescribing task will be noted in consecutive order on the sheet during an observation session to enable linking to the observed, time-stamped prescribing tasks recorded in WOMBAT.

Observers will undergo extensive training in using the data collection tool. Prior to starting data collection, inter-rater reliability (IRR) testing will be undertaken and data collection will only begin once observers have reached high agreement. IRR will be reassessed at several time points during the data collection period to ensure consistency over time between observers.

Prescribing errors

A senior clinical pharmacist working at the study hospital and providing clinical services to the ED will review all medication orders observed during the observation sessions in order to identify prescribing errors. The pharmacist will be provided with the date and time an order was written, as well as the prescribing doctor's name, which will aid in identifying the correct order for assessment.

Following the protocol used for previous prescribing error studies, errors will be classified into three broad categories: clinical, procedural and legal errors.37 38 Online supplementary appendix 1 shows the types of errors within each category and their definitions. Each medication order can have more than one error. For example, an order may use an unapproved abbreviation for the dose, as well as have a wrong dose error. To assess a medication order for clinical errors, the pharmacist will access the patient's clinical notes, including reason for their ED visit, clinical history and pathology results. Once prescribing orders are assessed by the clinical pharmacist for errors, a researcher with pharmacy training will enter the details into a Microsoft Access database, double checking the consistency of coding. Each patient's year of birth, drug name, route, dose and frequency will also be entered into the database. Details of clinical errors will be noted for verification by a professor of clinical pharmacology.

The severity of the potential harm of prescribing errors identified, except legal errors, will be independently assessed by the clinical pharmacist and pharmacy researcher and rated as follows: (1) insignificant, that is, incident is likely to have little or no effect on the patient; (2) minor, that is, incident is likely to lead to an increase in level of care (eg, review, investigations, or referral to another clinician); (3) moderate, that is, incident is likely to lead to permanent reduction in bodily functioning, increased length of stay, or surgical intervention; (4) major, that is, incident is likely to lead to a major permanent loss of function; (5) serious, that is, incident is likely to lead to death.39 In cases where there is disagreement between the severity codes assigned and one of the assessors has assigned a code of moderate or higher severity, a final severity code will be assigned through discussion and consultation with an experienced clinical pharmacist independent of the study and if required a professor of clinical pharmacology.

Definitions of multitasking and interruptions

For this study multitasking is defined as performing two tasks simultaneously (in parallel). An interruption is defined as a response to an external stimulus (prompt) which causes the participant to stop the task they are engaged in (the primary task), and switch to the secondary (interrupting) task. This may also be described as task-switching, a term used in the cognitive psychology literature.40 In this study, we will classify the nature of the observable external stimulus, which we refer to as a prompt. The type of prompt will be recorded as communication of a clinical, management or social nature (following the definitions in table 1), a phone call, public address (PA) announcement relevant to the observed doctor, equipment alarm or emergency alarm. If the prompt is communication, the person who initiated it will also be recorded.

Both multitasking and task-switching may be triggered by a prompt. For example, a nurse may approach a doctor while typing notes on a computer and ask about a patient's clinical management. This prompt may lead the doctor to stop typing and engage in a conversation with the nurse, which indicates he/she has chosen to task-switch. Alternatively the doctor may decide to respond to the prompt by speaking with the nurse while continuing to type, thus multitasking. Additionally, possible responses to external prompts which will be captured in the study include deferring the prompt, for example, by replying they will speak to the nurse in 5 min, deflecting the prompt by telling the nurse to ask another doctor, or acknowledging the nurse's statement or question with a short reply of ‘ok’. Both the prompt and response observed will be recorded in the WOMBAT system during the direct observations of doctors in the study. Recording the prompt and the responses in this way, allows for a more nuanced understanding of how doctors handle competing work demands, while maintaining comparability with previous study definitions.

Sample size and statistical analysis plan

During pilot testing, prescribing tasks were observed to occur about three times per hour with an average of two orders per prescribing task. Detectable effect sizes for differences in error rates were assessed for a sample size of 360 tasks (120 h of observation) across a range of plausible assumed values. Calculations were performed for multivariate Poisson regression with a main binary independent variable of interest (eg, was the prescribing task interrupted or not), assumed to occur in 50% of prescribing tasks, with the R2 of the other covariates assumed to be in the range 0.1–0.5. This is a slightly simplified scenario, compared to the proposed analysis, in order to facilitate the sample size calculation. Error rates were allowed to vary from 0.2 to 0.5 per order consistent with clinical ED errors previously reported at around 20%31 and assuming legal errors will be more common. With significance set at 0.05 and power at 0.8, the detectable increase in error rate due to either task-switching or multitasking ranged from 23% to 53%.

Analyses will be conducted at the prescribing task level, where each task may include multiple orders. The prescribing error rate for a given prescribing task is the number of errors divided by the number of medication orders. Prescribing error rates will be calculated for clinical, procedural and legal prescribing errors separately, as well as for all types of errors combined. Multivariate Poisson regression will be applied to the number of errors per task with the number of orders per task as the offset (ie, denominator), hence modelling the error rate at task level. If there is a preponderance of prescribing tasks with no errors then a multivariate zero-inflated Poisson model will be applied. The main covariates of interest will be rates of task-switching and multitasking per unit time. Since the number of errors, task-switches and multitasks are proportional to task length it is necessary to include all of these in the model as rates. We will also adjust for other factors including patient age, doctor seniority and patient load at the time of the prescription. Patient load data will be extracted from the ED information management system. A generalised estimating equations approach will be used to adjust SE estimates for clustering of outcomes within individual doctors. Sensitivity analyses will be performed in which task-switching and multitasking are included as the proportion of time spent performing a secondary task. Where sample size allows, a sub-analysis will apply the same Poisson model stratified by error severity.

Limitations

The ‘Hawthorne’ effect is when observed participants change their behaviour due to being under observation. Studies of this effect in clinicians suggest it is minimal,41 as does our prior research using direct observation of clinicians.35 However, observers will be trained and conduct extensive IRR testing in the ED prior to starting the study, which will ensure staff in the ED are accustomed to being observed and seeing the observers in the department. If doctors were to change their behaviour, either by minimising their responses to external stimuli or being more diligent in prescribing, this will lead to an underestimation of studied effects.

Factors associated with the study setting need to be acknowledged when considering the generalisability of the study findings. First, the study ED has paper based prescribing, but electronic patient records. Settings with electronic prescribing systems are likely to show lower prescribing error rates than those seen in our study.38 However, the evidence suggests that electronic prescribing is more likely to reduce the rates of procedural and legal errors, rather than clinical errors.38 Our analysis plans to examine all errors, and procedural, legal and clinical error rates separately, and thus will provide evidence for comparison to settings with electronic prescribing. Additionally, our study will be based in one ED in Australia and only observe doctors on day shift. However, errors due to interruptions and multitasking, occur due to an overload on cognitive processing, irrespective of settings. The focus on day shift will limit variability in errors due to fatigue, which is linked to decreased cognitive function. Thus, though the prescribing errors described in our study may not be the same across all ED settings, their relationship with interruptions or multi-tasking should be relevant to other settings. Naturally, the frequency of prompts, interruptions and multi-tasking may also vary across settings, but several studies of clinicians in the ED report these at consistently high levels.16 20 42 43

Ethics and dissemination

All participants meeting the inclusion criteria will be invited to participate and will be provided with a participant information sheet. Those agreeing to participate will provide written informed consent. Doctors participating in the study will be assigned a unique participant identifier. The list of doctors and identification numbers will be kept separate from the data collection. This data-key list will be destroyed at the end of the study. Any prescribing error identified that could result in serious patient harm will be followed up by the clinical pharmacist as per hospital incident procedures and reporting policies. No patient names will be recorded, all MRNs collected as part of the study will be destroyed on study completion, and a de-identified code will replace MRNs in the final dataset. Ethics clearance for the study protocol has been obtained from the hospital human research ethics committee (reference number 13/310).

The study results will be disseminated through scientific publications and presentations. Aggregate feedback will be given to the staff within the ED in relation to prescribing errors and patterns of work identified.