Multimodal system designed to reduce errors in recording and administration of drugs in anaesthesia: prospective randomised clinical evaluation.

OBJECTIVE: To clinically evaluate a new patented multimodal system (SAFERSleep) designed to reduce errors in the recording and administration of drugs in anaesthesia.
DESIGN: Prospective randomised open label clinical trial.
SETTING: Five designated operating theatres in a major tertiary referral hospital. PARTICIPANTS: Eighty nine consenting anaesthetists managing 1075 cases in which there were 10,764 drug administrations. INTERVENTION: Use of the new system (which includes customised drug trays and purpose designed drug trolley drawers to promote a well organised anaesthetic workspace and aseptic technique; pre-filled syringes for commonly used anaesthetic drugs; large legible colour coded drug labels; a barcode reader linked to a computer, speakers, and touch screen to provide automatic auditory and visual verification of selected drugs immediately before each administration; automatic compilation of an anaesthetic record; an on-screen and audible warning if an antibiotic has not been administered within 15 minutes of the start of anaesthesia; and certain procedural rules-notably, scanning the label before each drug administration) versus conventional practice in drug administration with a manually compiled anaesthetic record. MAIN OUTCOME MEASURES: Primary: composite of errors in the recording and administration of intravenous drugs detected by direct observation and by detailed reconciliation of the contents of used drug vials against recorded administrations; and lapses in responding to an intermittent visual stimulus (vigilance latency task). Secondary: outcomes in patients; analyses of anaesthetists' tasks and assessments of workload; evaluation of the legibility of anaesthetic records; evaluation of compliance with the procedural rules of the new system; and questionnaire based ratings of the respective systems by participants.
RESULTS: The overall mean rate of drug errors per 100 administrations was 9.1 (95% confidence interval 6.9 to 11.4) with the new system (one in 11 administrations) and 11.6 (9.3 to 13.9) with conventional methods (one in nine administrations) (P = 0.045 for difference). Most were recording errors, and, though fewer drug administration errors occurred with the new system, the comparison with conventional methods did not reach significance. Rates of errors in drug administration were lower when anaesthetists consistently applied two key principles of the new system (scanning the drug barcode before administering each drug and keeping the voice prompt active) than when they did not: mean 6.0 (3.1 to 8.8) errors per 100 administrations v 9.7 (8.4 to 11.1) respectively (P = 0.004). Lapses in the vigilance latency task occurred in 12% (58/471) of cases with the new system and 9% (40/473) with conventional methods (P = 0.052). The records generated by the new system were more legible, and anaesthetists preferred the new system, particularly in relation to long, complex, and emergency cases. There were no differences between new and conventional systems in respect of outcomes in patients or anaesthetists' workload.
CONCLUSIONS: The new system was associated with a reduction in errors in the recording and administration of drugs in anaesthesia, attributable mainly to a reduction in recording errors. Automatic compilation of the anaesthetic record increased legibility but also increased lapses in a vigilance latency task and decreased time spent watching monitors. Trial registration Australian New Zealand Clinical Trials Registry No 12608000068369.

RCT Entities:   Population Interventions Outcomes

Introduction

Iatrogenic harm is an important public health problem with high human and financial costs.1 2 In anaesthesia, errors in drug administration are particularly problematic, can occasionally have devastating consequences for patients, and ought to be preventable.3 4 5 6 7 There have been calls for action in this regard, and in the United States the Anesthesia Patient Safety Foundation recently held two summit meetings to address this problem.8 9 10 The importance of errors in drug administration in other specialties within healthcare is also well recognised.11 12 13 14

The anaesthetic record is an important clinical tool for decision making during anaesthesia and postoperatively. It is an important document for legal purposes and a source of data for research, audit, and continuous quality improvement. Inaccuracies in the recording of administered drugs can lead to subsequent errors, such as the repeated administration of a dose of drug given but not noted. This applies not only in anaesthesia but also in other settings.12 Anaesthetic records are traditionally compiled by hand, a process that accounts for about 15% of an anaesthetist’s intraoperative time.15 16 Handwritten records are often incomplete and inaccurate,6 17 18 which might reduce or negate their value for any of the above purposes.

To enhance safety and quality through system improvement, a set of strategies designed to reduce errors in the administration and recording of drugs given during anaesthesia and facilitate and improve the quality of anaesthetic record keeping has been incorporated into a novel system (SAFERSleep: Safer Sleep LLC, Nashville, TN).19 20 21 The new system is in use in several hospitals in New Zealand and the United Kingdom and, in particular, has been used for most anaesthetics at Auckland City Hospital since mid-2005. Certain aspects of the system have been evaluated in other settings—for example, several studies have shown that the use of bar coding improves drug safety.12 13 14 22 23 There have also been numerous studies of electronic record keeping in anaesthesia and in healthcare generally. The value of electronic health records, however, has not been fully elucidated, and few relevant studies have used a prospective randomised design.24 25 26

Many apparently effective safety innovations have been introduced in medicine in an attempt to reduce error or improve quality. Most innovations of this type come at some cost, and the new system is no exception.21 As a minimum this cost represents an opportunity loss in respect of other possible uses of limited healthcare resources.27 More worryingly, unintended (and perhaps undesirable) consequences are also a possibility with any new process.28 29 We conducted a prospective randomised open label clinical trial to evaluate the new system’s impact on errors in the recording and administration of drugs in anaesthesia and on anaesthetists’ vigilance and workload.

Methods

Hypotheses

Our primary (null) hypothesis was that there would be no significant difference between the new system and conventional methods in the rate of errors in the recording and administration of medications or in the number of lapses in responding to a vigilance latency test. We also hypothesised that there would be no differences in compliance with the need to administer prophylactic antibiotics within one hour of incision, in outcomes in patients, in anaesthetists’ workload (either self recorded or recorded by observers), in the preferences of participating anaesthetists, or in the legibility of the anaesthetic records. Finally, we hypothesised that there would be no difference between Māori and other patients in our primary outcome variables or in anaesthetists’ workload.

Trial design and participants

The study protocol and registration details were updated twice before the finalisation of data entry and subsequent analysis of results to add the terms “iv” and “iv drug” for clarification and because it became apparent that components of the combined primary outcome variable, as initially defined, had different denominators and could not be added to each other. Data were collected from five designated operating theatres in the adult anaesthetic departments of Auckland City Hospital. Participating anaesthetists were familiarised with the trial and its objectives and gave written informed consent before taking part. All participating anaesthetists had received formal training in the use of the new system before the start of this study. Before surgery, all patients scheduled for anaesthesia in the designated study theatres were given an information sheet that informed them of the nature of the study and the consequent presence of an observer during their surgery and that they could decline to participate in the study. For clarity, we have used the term “case” throughout to mean one patient undergoing anaesthesia for surgery on one occasion.

The study’s statistician (CF) performed randomisation by week, with treatment allocation codes in blocks of four and with stratification for study theatre, with a computer generated random sequence (Microsoft Excel, Redmond, WA). Theatres were set up for provision of anaesthesia with either the new system or conventional methods according to the randomisation schedule at the start of each week and remained so for that week. The operating theatres selected and surgical specialties represented were chosen to give a broad case mix. No attempt was made to manipulate the scheduling of anaesthetists to study theatres. Observers attempted to contact anaesthetists the day before each study day to alert them to the study and (if they asked) the system under which they would be working. Anaesthetists could decline to participate on any given day without retracting their overall consent.

Interventions

The new system has previously been described in detail.20 It includes the following elements:

Customised drug trays designed to promote a well organised anaesthetic workspace and aseptic technique

Pre-filled syringes for the most commonly used anaesthetic drugs (see table A in appendix on bmj.com for details)

Large legible drug labels, colour coded according to standards adopted in several countries30 31 32 33

A barcode reader linked to a computer, speakers, and touch screen to provide automatic auditory and visual verification of the selected drug immediately before each administration and to record its administration (keyboard entry of drug name is possible as an alternative)

An on-screen visual and auditory warning if an antibiotic has not been administered within 15 minutes of the start of anaesthesia

Purpose designed drug trolley drawers

Automatic compilation of an anaesthetic record, on screen in real time and as a paper print-out at the end of anaesthesia.

The following six procedural rules are followed for the correct use of the system:

Appropriate organisation of syringes and ampoules

Ampoules appropriately laid out at the start of the case

All empty ampoules and syringes retained for reconciliation

All syringes labelled

Computerised voice prompt audible throughout each anaesthetic

Medication label scanned before each drug administration.

Of these, the last two principles are considered the most crucial to the effectiveness of the new system.

The conventional management option included the following elements:

A standard drug tray to hold the syringes and ampoules

A standard fully stocked drug trolley

All drugs drawn up by the anaesthetist

Small standardised colour coded drug labels, to be applied by the anaesthetists

Standard anaesthetic record chart to be filled in by hand, with usual access to data routinely logged by the anaesthetic monitor if desired.

Data collection

Trained observers collected data in the operating theatre. Training included reading an introductory textbook of anaesthesia,34 observing at least five cases, an emphasis on discreet observation with minimal unnecessary conversation, and collecting data simultaneously with an experienced trained observer for a minimum of 10 cases or until acceptable concordance was achieved. An investigator experienced in observational research (RH) judged competence to collect actual study data independently. For the first 416 cases the observers were present throughout to collect workload and task analysis data. For the remainder they were present for most, but not necessarily all, of each case. Observers were, however, present in the operating theatre at the beginning and end of every case. Two investigators (RH and PR) with no relevant conflict of interest were explicitly asked to oversee the study processes; among other things they made several visits to the study theatres to personally inspect the processes of observation and data collection. Two overseas collaborators with no relevant conflicts of interest visited Auckland City Hospital during the study to inspect the data collection processes (again by visiting the operating theatres) and other aspects of the study.

Recording and administration of drugs (reconciliation process)—To facilitate identification of errors in the recording and administration of drugs, we took a full inventory of the contents of the anaesthetic drug drawers before each case. Anaesthetists were asked to retain all syringes and vials and to avoid discarding unused contents of syringes. Empty sharps bins were used to facilitate counting of any drugs inadvertently discarded during the case. At the end of each case, the drugs used and the total doses given were identified through these means and through reconciliation of the remaining contents of the drug drawers against the preoperative inventory. These drug names and total doses were compared with the drug administrations recorded on the final anaesthetic record and any discrepancies noted.

Vigilance—We used a vigilance latency task to assess anaesthetists’ vigilance and spare work capacity.35 This task required the anaesthetists to acknowledge the illumination of an easily visible bright light on the anaesthetic machine, which appeared at random intervals of nine to 14 minutes. If it was not acknowledged within 300 seconds, the light went out and the failure to respond was classed as a lapse of vigilance. Vigilance data were recorded with customised software operating on a personal digital assistant (Hewlett Packard iPAQ, Houston, TX). The vigilance latency task was manually activated when the anaesthetist entered the operating theatre at the start of each case and continued until the anaesthetist left with the patient.

Task analysis and workload assessment—All tasks performed by anaesthetists and anaesthetic trainees in the operating theatre were directly coded and entered on to a computer with a custom written task analysis program, based on the work of Weinger36 35 but modified to focus on tasks specifically relevant to drug administration and anaesthetic record making. The program calculated the duration of defined tasks from start and stop times entered via touch screen by the observers. Additional free text descriptions and comments were possible. Observation began when the anaesthetist(s) entered the operating theatre at the start of each case, included preparation before entry of the patient, and continued until departure of the patient at the end of the case. At random intervals of between seven and 15 minutes, the computer program prompted recordings of perceived workload with the Borg workload score.37 This score is a 15 point interval scale between six and 20, where 12 is the workload of a normal tracheal intubation. Workload scores were assessed independently by the observing researcher and self reported by the anaesthetist.

Legibility—As a measure of legibility we examined the first and last 100 records in each arm of the study and recorded the total time to extract the number of discrete blood pressure and temperature measurements recorded; the number of intravenous drugs recorded; and the number of bolus doses of medication recorded.38

Compliance—At the end of each case the observer assessed the anaesthetists’ compliance with the procedural rules as described above for the correct use of the new system. Each item was scored as follows: yes=2, sometimes=1, no=0. The scores were summed and the result expressed as a percentage of the maximum possible score.

Ratings—Anaesthetists who managed 10 or more study cases were asked to rate the new system and conventional methods in respect of 18 attributes and six case types with 100 mm unmarked visual analogue scales with appropriate anchors (indicating dissatisfaction at 0 mm and maximum satisfaction at 100 mm).

Ethnicity—Ethnicity of patients was recorded.

Outcomes

Primary outcomes

The primary outcome variable was the composite of errors in recording or administering intravenous drugs. Errors in drug administration included giving the incorrect drug (substitution) and failing to give the intended drug (omission). Errors in drug recording included failing to record an administered drug; recording an administered drug name but failing to record the dose; and discrepancy between the recorded total dose and the dose calculated through the reconciliation process by observers. A panel of four anaesthetists blinded to the treatment arm in which the discrepancies occurred evaluated dose discrepancies. For each drug, consensus was reached on the minimum discrepancy that would be clinically relevant in the context of the dose typically used for each drug in Auckland City Hospital and of the pharmacology of the drug (see table B in appendix on bmj.com). A discrepancy in dose was deemed clinically relevant, and therefore counted, when the total dose documented by the anaesthetist on the final anaesthetic record differed from that documented by the observer by this amount or more. In cases where there was unresolved inconsistency between the identities of the reconciled ampoule, the labelled syringe, and the drug recorded by the anaesthetist the error was classified as “incorrect label.” The composite primary outcome variable and the two individual error types were quantified as the number of errors per administration for each case and represented as overall rates per 100 administrations.

Secondary outcomes

Patient outcomes—We reviewed the hospital records of all patients after their discharge and recorded the following predefined adverse outcomes: anaphylaxis, myocardial infarction, sepsis, septic shock, or death. We also sought any evidence of harm attributable to a drug error. In addition, for each case we determined whether antibiotics would normally be required as a matter of surgical policy at the hospital and, if so, whether they were administered within the 60 minutes before surgical incision.

System outcomes—Task analysis and workload assessments, legibility of anaesthetic records, compliance with procedural rules of the new system, and questionnaire ratings were analysed as described below.

Statistics

The study statistician (CF) analysed the data with SPSS version 15.0 (SPSS, Chicago, IL). A subset of data from 2.5% of the cases, selected randomly, was re-entered to check the accuracy of the data entry. A two tailed probability of P≤0.05 was predefined as indicative of significance. The primary null hypothesis—that there would be no significant difference between the new system and conventional methods in errors in recording or errors in administration of drugs—was tested with a general linear mixed model that included “operating theatre week” as a random effect. The number of errors per administration was calculated for each case. This involved summing all the relevant errors for the case and dividing by the total number of administrations for that case. The unit of analysis was the case and the unit of randomisation was the operating theatre week. The results are reported as errors per 100 drug administrations. The effects of patients’ American Society of Anesthesiologists (ASA) score, type of anaesthetic procedure (emergency/elective), ethnicity, sex, and age on error rates were also assessed with a general linear mixed model that simultaneously tested all these effects.

We compared the proportion of anaesthetists experiencing lapses in vigilance between randomised groups using χ2 tests or Fisher’s exact tests when the expected error rates were low. Total times spent on individual and collective tasks, as recorded by observers, and the mean workload assessments were also compared between randomised groups with general mixed linear models that included “operating theatre week” as a random effect. The appropriateness of these parametric models for testing the primary and secondary outcomes was confirmed with normality plots of model residuals. Spearman’s rank correlation coefficients were used to assess the strength of the relations between the error rate and anaesthetist’s self reported workload, and the error rate and compliance scores for use of the new system, per case. Workload scores as assessed by the observer were correlated with anaesthetists’ self reported workload scores with Spearman’s correlation coefficients. The total time spent extracting information as an assessment of legibility of the anaesthetic record was compared between randomised groups with an independent t test.

We calculated mean visual analogue scale ratings for the new system and for conventional methods for each anaesthetist over the 18 attributes and the six scores relating to case types (that is, a mean of the 24 items) and compared them between randomised groups with a paired t test. Differences in individual items were evaluated post hoc (given significance in the overall comparison) with paired t tests with a Bonferroni correction to allow for multiple testing.

Sample size estimation

No comparable data were available for observational research on errors in recording or administering drugs in the clinical setting, though facilitated incident recording has identified one error in every 133 cases and an average of about 10 drug administrations per case.3 6 In observational research with a human patient simulator, the composite error rate for recording and administration of drugs was about 0.3 per administration,6 but this study was designed to greatly increase the likelihood of error. If one assumed one error per 10 administrations in the control group, about 3300 administrations (that is, about 330 cases) per group would be needed to show a 20% reduction with 80% power and a two tailed α=0.05. This calculation is based on the binomial distribution of the number of errors per case, allowing for 10 administrations per case and then using this to estimate variation between cases in the error rate per administration. Given the lack of clear data to support these assumptions, and taking into account the pragmatic limitations of our study and uncertainty over the assumption of independence inherent in this estimate, we aimed to study at least 500 cases in each study arm. Because of the intensive nature of the task analysis data collection and the greater sensitivity of these continuous data, the collection of this information was ultimately limited to about the first 400 cases.

Results

Data were collected from five designated operating theatres in Auckland City Hospital between March 2008 and February 2009, at which time recruitment targets were met. A total of 1748 cases were managed in the study theatres during the time of the trial; two were excluded because the patients indicated a preference not to participate and 502 because no observer was available; 613 of the eligible cases were allocated to the new system and 631 to conventional methods. On the day, anaesthetists declined to participate in 46 (new system) and 111 (conventional) of these study cases respectively (P<0.001). One record made by the new system was lost because of a computer malfunction and 11 made by conventional methods could not be retrieved from the hospital’s record system so data were analysed from 1075 cases (566 with the new system and 509 with conventional methods, figure). There was 100% concordance between the records in the primary dataset and the 2.5% of records re-entered for quality control.

Flow of cases (patients undergoing anaesthesia for surgery) through trial; five operating theatres randomly allocated by week to one or other method for 83 study weeks

Anaesthetists, patients, and cases

Eighty nine anaesthetists took part in the study, including 54 specialists and 35 trainees. The specialists had a median of 10 years’ (range 1-35) experience. Nine anaesthetists refused to participate at all, and five withdrew completely after limited participation. Some anaesthetists refused to participate on particular days; reasons for this included not wanting to use the conventional manual record for complicated patients (none cited not wanting to use the new system as a reason), reluctance to deal with the study while teaching medical students, finding participation onerous, and disliking taking part in the study.

Characteristics of patients and the mix of surgical specialties, emergency/elective cases, ASA scores, and duration of anaesthesia were well balanced between the two groups (table 1).

Table 1

 Patients’ characteristics, ASA score, duration of anaesthesia, case type (emergency/elective), and surgical specialty for participating patients in study comparing new system designed to reduce errors in recording and administration of drugs in anaesthesia and conventional methods. Figures are numbers (percentage) of patients unless otherwise specified

	New system (n=566)	Conventional methods (n=509)
Mean (SD) age years	57 (19)	57 (19)
Men	287 (51)	249 (49)
Ethnicity:
European	370 (65)	322 (63)
Māori	43 (8)	52 (10)
Pacific Island	69 (12)	52 (10)
Other	84 (15)	83 (16)
ASA score:
1	137 (24)	112 (22)
2	228 (40)	200 (39)
3	170 (30)	169 (33)
4	30 (5)	27 (5)
Not recorded	1 (0)	1 (0)
Emergency case	214 (38)	222 (44)
Duration of anaesthesia (hours):
<1	273 (48)	236 (46)
1-2	192 (34)	185 (36)
2-3	63 (11)	49 (10)
≥3	38 (7)	39 (8)
Surgical specialty:
General surgery	175 (31)	174 (34)
Orthopaedic	192 (34)	135 (27)
Urology	159 (28)	171 (34)
Cardiothoracic	36 (6)	28 (6)
Other	4 (1)	1 (0)

ASA=American Society of Anesthesiologists.

The total number of drug administrations with the new system and conventional methods was 5680 and 5084, respectively, equating to an average of about 10 drug administrations per case for both.

Main outcomes

Errors

There were 471 errors in drug administration or recording from 5680 administrations in 566 cases with the new system and 488 errors from 5084 administrations in 509 cases with conventional methods. The mean rate of these errors per 100 administrations, calculated from the general linear mixed model, was 9.1 (95% confidence interval 6.9 to 11.4) with the new system (one in 11 administrations) and 11.6 (9.3 to 13.9) with conventional methods (one in nine administrations) (P=0.045), a reduction in the error rate of 2.5 per 100 administrations (0.3 to 4.8) or 21% (table 2). The rate of errors in drug administration per 100 administrations was 0.16 with the new system (one in 625) and 0.33 with conventional methods (one in 303; P=0.120). The rate of recording error per 100 administrations was 9.68 with the new system (one in 10) and 12.10 with conventional methods (one in eight; P=0.049).

Table 2

 Error rates per 100 drug administrations for new system designed to reduce errors in recording and administration of drugs in anaesthesia (5680 administrations) and for conventional methods (5084 administrations)

	New system (n=566 cases)	Conventional methods (n=509 cases)	P value
Overall error rate* (95% CI)	9.1 (6.9 to 11.4)	11.6 (9.3 to 13.9)	0.045
Errors in drug administration:
Incorrect drug given (substitution)	0.03	0.18	0.09
Drug not given (omission)	0.11	0.14	0.63
Errors in drug recording:
Drug given, not recorded at all	2.31	3.50	0.002
Drug given, dose not recorded	0.05	0.67	<0.001
Discrepancy between recorded and observed total dose	6.86	7.18	0.94
Incorrect label†	0.02	0.04	0.67

*Derived from general linear mixed model, which included operating theatre as random factor, so might differ from error rate that can be derived from table 5.

†Inconsistency between reconciled ampoule, labelled syringe, and drug recorded by anaesthetist. We were unable to identify in this situation whether intended drug was that recorded, given, or on label.

Errors in drug recording and administration were more common in male than in female patients, but there were no significant differences between operating theatres, ASA scores, elective/emergency cases, or ethnic groups (table 3), and error rate was not related to patient’s age. When indicated, antibiotic prophylaxis was given before incision in 94% of patients with the new system and 93% with conventional methods (P=0.48).

Table 3

 Error rates* per 100 drug administrations by demographic group of patients, ASA score, and emergency/elective case

	Mean (95% CI)	P value
Ethnicity:
European	10.2 (8.3 to 12)	0.62
Māori	11.6 (7.4 to 15.8)
Pacific Island	8.5 (4.7 to 12.3)
Other	11.3 (8.0 to 14.5)
Sex:
Male	11.7 (9.5 to 14.0)	0.03
Female	9.0 (6.7 to 11.3)
ASA score:
1	9.2 (6.5 to 12.0)	0.63
2	11.0 (8.8 to 13.2)
3	11.3 (8.9 to 13.7)
4	10.0 (4.7 to 15.2)
Surgery type:
Emergency	10.6 (8.2 to 13.0)	0.78
Elective	10.2 (8.0 to 12.4)

ASA=American Society of Anesthesiologists.

*Derived from general linear mixed model, which included operating theatre as random factor, so might differ from those that can be derived from table 5. When included as a covariate, age was not significantly associated with error rate (P=0.13). There was no significant difference between operating theatres (P=0.4).

Vigilance

Evaluable data were obtained for 944 cases. Lapses in response to the vigilance latency task occurred in 12% (58/471) of cases with the new system and 9% (40/473) with conventional methods (P=0.052, χ2 test).

Secondary outcomes

Patients’ outcomes

No errors identified in this study caused any harm to any patient. Nine patients in each group experienced a major complication (anaphylaxis, myocardial infarction, sepsis or septic shock). Five patients died during their hospital admission in the group anaesthetised with the new system compared with six in the conventional management group.

Task analysis and workload assessment

Task analysis and workload assessment data were collected for 416 cases. Less time was spent record keeping with the new system than with conventional methods and slightly less time observing monitors; on the other hand more time was spent observing patients and preparing and administering drugs (table 4).

Table 4

 Mean (SE) times spent on various tasks in study comparing new system designed to reduce errors in recording and administration of drugs in anaesthesia with conventional methods

Durations	New system (n=208 cases)	Conventional methods (n=208 cases)	P value*
Drug preparation	15 min 30 sec (29 sec)	14 min 23 sec (29 sec)	0.06
Drug administration†	9 min 2 sec (18 sec)	8 min 2 sec (18 sec)	0.01
Establishing vascular access and establishing/adjusting IV infusions	10 min 36 sec (30 sec)	9 min 36 sec (30 sec)	0.10
Airway procedures	9 min 45 sec (38 sec)	9 min 43 sec (39 sec)	0.96
Adjusting anaesthetic equipment	10 min 12 sec (33 sec)	9 min 47 sec (33 sec)	0.53
Positioning patient	2 min 42 sec (9 sec)	2 min 29 sec (9 sec)	0.24
Observing monitors	20 min 22 sec (63 sec)	23 min 1 sec (64 sec)	0.04
Observing patient	28 min 26 sec (84 sec)	21 min 22 sec (85 sec)	<0.001
Recording	10 min 23 sec (41 sec)	20 min 5 sec (42 sec)	<0.001
Tidying workspace	3 min 28 sec (10 sec)	3 min 17 sec (10 sec)	0.33
Task and teaching related conversation	33 min 42 sec (79 sec)	31 min 27 sec (80 sec)	0.16
Task related reading	1 min 41 sec (12 sec)	1 min 57 sec (13 sec)	0.30
Other tasks	6 min 10 sec (31 sec)	7 min 1 sec (31 sec)	0.17

*Calculated by analysis of variance (ANOVA) with total task duration and theatre as covariates in general linear mixed model.

†Drug administration included administering IV (intravenous) drugs by bolus and infusion and adjusting vapourisers and gas flow.

The mean self reported Borg workload scores for the two systems were identical, at 13.2 (12.8 to 13.6). Patients’ ethnicity had no effect on anaesthetists’ workload. There was a strong correlation between these self reported scores and the observer ratings (r=0.76, P<0.001), but anaesthetists’ ratings were 5% higher than the observers’. The error rate per drug administration correlated with the anaesthetists’ mean rating of workload for each case (r=0.15, P=0.004).

Legibility

It took a mean time of 2 min 18 sec (SE 4 sec) to extract the three specified pieces of information from each record made with the new system and 3 min 41 sec (8 sec) from each record made by the conventional system (P<0.001).

Compliance with principles of the new system

The mean compliance score for cases in which the new system was used was 81% (SE 0.8%, range 40-100%). The proportion of cases (from 456 with evaluable ratings) in which anaesthetists complied with individual principles of the new system were 78% for appropriate organisation of syringes and ampoules; 89% for ampoules appropriately laid out at the start of the case; 92% for all empty ampoules and syringes retained for reconciliation; 84% for all syringes labelled; 79% for computerised voice prompt audible throughout the case; and 62% for medication label scanned before each drug administration.

Anaesthetists were fully compliant with all the principles of the new system in only 83 of 456 cases (18%). Compliance score was negatively correlated with error rate (r=−0.11, P=0.016), with 42% of the variation among compliance scores attributable to differences between anaesthetists. The mean error rate per 100 administrations was 6.0 (3.1 to 8.8) when the anaesthetist complied with the two key principles of scanning the drug barcode before administering each drug and keeping the voice prompt active, and 9.7 (8.4 to 11.1) when the anaesthetist did not comply with them (P=0.004).

Rating scales

Twenty eight anaesthetists participated in 10 or more study cases; of these 21 (75%) completed a rating sheet for both the new system and conventional methods. The mean rating for all questions was 74 mm (SE 2 mm) for the new system and 60 mm (3 mm) for conventional methods; the mean difference was 14 mm (5 mm to 22 mm; P=0.003). After Bonferroni correction, there were significant differences in ratings, all favouring the new system, in the following scales: overall satisfaction (P<0.01), effort required (P=0.02), usefulness (P=0.05), and completeness (P=0.01) of the anaesthetic record generated by this method and attitude to use of the system for all cases (P=0.002). Differences in ratings for reliability of the system, its inherent safety as an approach to anaesthesia, time available for monitoring patients, attitude to advocating use in another institution, and the usefulness and safety of the drug trays, syringe labelling, methods for checking drug identity, and design of the drug trolley did not reach significance. The new system was preferred over conventional methods for long (P<0.01), complex (P<0.01), and emergency (P=0.02) cases; differences in ratings for short, simple, and routine cases were not significant.

Discussion

In comparison with conventional methods, use of a new system designed to reduce errors in anaesthesia was associated with a 21% reduction in the rate of errors in the recording and administration of drugs, more legible anaesthetic records that were compiled in half the time, and more time devoted to observing patients. On the other hand, less time was spent observing monitors, and there were more lapses in a vigilance latency task. Evaluations indicated an overall preference for the new system over conventional methods but emphasised the reduction in effort required and the completeness and usefulness of the records rather than the safety aspects of the system. This preference was also reflected by the number of cases in which anaesthetists declined to participate in the study on the day: 46 with the new system compared with 111 with conventional methods (P<0.001). Preferences for the new system were most evident for long, complex, and emergency cases. Slightly more time was spent drawing up and administering drugs with the new system. There was no difference in patients’ outcome, and no harm was caused by any of the errors observed in this study. No difference in self rated workload was seen between study arms.

There is international evidence of substantial ethnic disparities in the quality of healthcare39 and some evidence that such disparities might also apply to iatrogenic harm.40 We therefore took the opportunity to explore the possibility of ethnic disparity in the delivery of anaesthesia with a view to perhaps designing a future study with the primary aim of investigating this issue. In fact we found no evidence of any difference between the main ethnic groupings evaluated.

Our findings were broadly consistent with previous evaluations of the new system. Earlier studies found that drug preparation times were reduced but that participants had access to a greater number of prefilled syringes.21 23 Our rate of errors in drug administration was higher than previous estimates by our group using facilitated incident reporting3 22 and other studies that used standard incident reporting.4 41 This finding suggests that many anaesthetic errors remain unidentified by the person who makes them and that incident reporting tends to underestimate the problem.

Clinical importance of findings

Errors in drug administration are an ongoing source of concern in anaesthesia8 9 and in healthcare generally.12 The difference in administration errors was not significant, and no harm was identified from any of our errors, but we were not able to power our study for these end points. Our group has previously shown a significant reduction in drug administration error in incident reports from cases using the new system compared with those using conventional methods in over 74 000 anaesthetics.22 The potential for harm from drug administration error is substantial,4 5 though some are more hazardous than others—for example, the risk of substituting a drug for one of the same class (such as one opioid for another) is likely to be lower than the risk of substituting a drug for one of a different class, and the consequences of mislabelling or omitting a drug depends on the drug involved (table 5). Recording errors, although perhaps less likely to cause immediate harm to a patient, also have potential to impact on care particularly in the post-anaesthetic care unit and when patients are handed over from one anaesthetist to another. We have included only those dose discrepancy errors evaluated as clinically important, and there were many more that were discarded as trivial but that, nevertheless, were undesirable. It might be thought self evident that electronic records would be more accurate than manual ones,25 42 but the recording errors analysed in our study involved information that had to be entered interactively, so this was not a foregone conclusion.25 42

Table 5

 Specific drug administration errors and incorrect label errors in study comparing new system designed to reduce errors in recording and administration of drugs in anaesthesia with conventional methods. Number of times each event occurred is given in brackets

Error type (type total)	New system	Conventional methods
Substitution* (9):
Interclass (1)	—	Glycopyrrolate for atracurium (1)
Intraclass (8)	Atracurium for vecuronium (1); remifentanil for alfentanil (1)	Cefoxitin for cefuroxime (1); cephazolin for cefoxitin (3); cephazolin for cefuroxime (1); glycopyrrolate for atropine (1)
Omission (13)	Antibiotic† (1); atropine (2); gentamicin (1); heparin (1); morphine (1); salbutamol (1)	Dexamethasone (1); droperidol (1); metaraminol (2); morphine (1); vecuronium (1)
Incorrect label (5)	Atracurium labelled vecuronium (1); atracurium labelled rocuronium (1)	Glycopyrrolate labelled atropine (2); incorrect dilution (1)
Total	11	16

*Substitution errors between (interclass) and within (intraclass) pharmacological classes as defined by international colour code for anaesthetic drugs.

†No antibiotics given.

The higher number of lapses in the vigilance latency task with the new system was unexpected. Using a similar test, Weinger and colleagues showed no significant difference in vigilance latency between groups keeping manual or electronic records and concluded that electronic record keeping did not affect workload or vigilance.43 Our study is much larger, and we evaluated lapses because we thought that a failure to respond for five minutes or more would matter if the missed information were clinically relevant. Anecdotally, there was widespread scepticism among participants over the clinical relevance of the vigilance latency task, but impact on vigilance is a controversial aspect of electronic record keeping and our finding suggests that further investigation of this potentially negative effect is warranted. Ideally, future work should evaluate the times to respond to important changes in patients’ physiological status.

Our findings in relation to compliance with the systems’ principles are of interest. There was no difficulty getting anaesthetists to use the recording aspects of the system and to comply with some of its safety principles (the mean compliance score was 81%) but, despite ongoing efforts to provide education and motivation in this regard, full compliance with these principles occurred in only 18% of cases. Notably, this subset of participants, who engaged strongly in using the system as it was designed to be used, made significantly fewer errors. Evaluative ratings emphasised other perceived advantages of the system rather than its potential contribution to safety. Difficulty has also been found in engaging clinicians in other safety initiatives, notably in relation to hand washing.44 This might reflect complacency about the problem of iatrogenic harm in healthcare in general or about errors in drug administration in particular or perhaps scepticism about the principles of the system. We think the data on the importance of drug administration error in anaesthesia are compelling3 9 45 and hope that the negative correlation between compliance with the principles and the rate of error seen in this study will encourage greater compliance in future.

Weaknesses of the study

The study was carried out in a single centre in which the system had been in use for five years, with a sample of convenience; this might limit the degree to which its findings apply in general (that is, its external validity). There might have been various observer effects, but these would presumably have applied to both groups.46 47 We could not blind the study. In fact, even concealment proved impractical as several participants indicated early in the study that they preferred to receive forewarning of the method they were to use on any particular day. Nine anaesthetists declined to participate in the study at the outset, and five more withdrew early in the study (14%). On the day, anaesthetists declined to participate in 46 cases randomised to the new system and 111 cases randomised to conventional methods (13%). Anaesthetists were less likely to consent to taking part in the study when anaesthetising complex cases, when there was a preference for using the new system. This lack of equipoise in attitudes to the new system was a potential source of bias. Because of the occasional unavailability of our observers, we were unable to collect data on all cases. This too could have introduced bias, although it involved no apparent systematic element. Because of limited resources we were able to collect data only from scheduled operating lists, involving predominantly elective cases during working hours, so no inference is possible in relation to emergency cases done out of hours. We had difficulty categorising five errors in which the label on the syringe referred to one drug (such as vecuronium), the empty ampoule contained another (such as atracurium), and the drug recorded was the one in the ampoule (in this case atracurium). Some might argue that these events were not errors, but they reflect poor practice, and there was no objective way of being sure what the anaesthetist actually intended in each case. These events were evenly distributed (that is, 2:3) between the two arms of the study so have little potential to bias our results. In those that occurred during use of the new system the barcode scanning check was probably not used. Imperfect compliance with the system’s principles might also have reduced the effect size seen in our primary outcome variable from that potentially achievable with its correct use. The study was too small to investigate administration errors or harm but is by far the largest of its type (that is, randomised prospective investigation of the problem of drug administration error with data collection by observers) investigating this problem that we know of. It is not clear why errors were more common in male than in female patients, but study groups were well balanced in this regard so this difference is unlikely to have biased our results. The conflict of interest of some investigators was a potential source of bias. This risk was explicitly addressed by the inclusion of senior co-investigators with no conflicts of interest and by asking independent overseas collaborators to visit and review the study processes. In addition, none of the investigators with conflicts of interest were involved in the analysis of the data and the study was registered and independently funded.

Future research

Our results suggest room for improvement in the accuracy of drug recording and administration in anaesthetic practice, whether with the new system or with conventional methods. Our findings also provide objective evidence for the efficacy of the principles and rules underlying the system as part of the ongoing iterative process of its continuous improvement. Many of these are not protected but simply a matter of good practice (even checking before administration could be carried out with a second person instead of the computer and barcode reader48), so they could be applied immediately by anyone. Extension of these principles to drug administration in other areas of healthcare would be worth exploring. Qualitative research into the human and design factors that influence compliance with the system’s safety principles could be worthwhile. It might also be worth investigating the effectiveness of role models and of various educational and motivational approaches in this regard, particularly in relation to doctors in training.

Conclusions

We have added to previous evidence20 23 21 22 48 that supports the efficacy and clinical use of a multimodal approach to improving drug administration and record keeping in anaesthesia, which includes the use of barcodes and computers in checking the identification of drugs before administration. We have shown that this approach reduces a composite end point of errors in the recording and administration of drugs, increases the legibility and reduces the time spent creating anaesthetic records, and increases time spent observing patients during anaesthesia. Conversely, it decreases time spent watching monitors and increases lapses in a vigilance latency task. Compliance with the systems’ principles evaluated in this study was inversely related to error rates, but achieving high levels of compliance was difficult. This study adds to the evidence informing effective system redesign for drug administration and record keeping in anaesthesia, and potentially other areas of healthcare as well. It is a step forward in an ongoing process of continuous quality improvement in anaesthesia.

Errors in the recording and administration of drugs are common in anaesthesia specifically and healthcare generally and can lead to poor care and harm patients

Checks with barcodes and computers before administration of drugs are feasible and can reduce such errors, although there are few randomised trials to show this

In a prospective randomised controlled trial we showed that a multimodal approach decreased errors in recording or administering drugs, reduced the time spent in keeping records, and improved their legibility; more time was spent watching patients

There were more lapses in a vigilance latency task and less time was spent watching monitors

Compliance with principles of safe drug administration was inversely associated with error rates, but achieving high levels of compliance with the safety principles of drug administration was difficult