Training in intraoperative handover and display of a checklist improve communication during transfer of care: An interventional cohort study of anaesthesia residents and nurse anaesthetists.

BACKGROUND: Handovers during anaesthesia are common, and failures in communication may lead to morbidity and mortality.
OBJECTIVES: We hypothesised that intraoperative handover training and display of a checklist would improve communication during anaesthesia care transition in the operating room.
DESIGN: Interventional cohort study.
SETTING: Single-centre tertiary care university hospital. PARTICIPANTS: A total of 204 random observations of handovers between anaesthesia providers (residents and nurse anaesthetists) over a 6-month period in 2016. INTERVENTION: Two geographically different hospital sites were studied simultaneously (same observations, but no training/checklist at the control site): first a 2-week 'baseline' observation period; then handover training and display of checklists in each operating room (at the intervention site only) followed by an 'immediate' second and finally a third (3 months later) observation period. MAIN OUTCOME MEASURES: A 22-item checklist was created by a modified DELPHI method and a checklist score calculated for each handover by adding the individual scores for each item as follows: -1, if error in communicating item; 0, unreported item; 0.5, if partly communicated item; 1, if correctly communicated item.
RESULTS: Before training and display of the checklist, the scores in the interventional and the control groups were similar. There was no improvement in the control group's scores over the three observation periods. In the interventional group, the mean (95% confidence interval) score increased by 43% [baseline 7.6 (6.7 to 8.4) n = 42; 'immediate' 10.9 (9.4 to 12.4) n = 27, P < 0.001]. This improvement persisted at 3 months without an increase in the mean duration of handovers.
CONCLUSION: Intraoperative handover training and display of a checklist in the operating room improved the checklist score for intraoperative transfer of care in anaesthesia.

Introduction

Most preventable adverse events in medicine are because of communication errors, and more than half of these occur in relation to transitions of care. Hudson et al.[1] found a 43% increase of in-hospital mortality and a 27% increase of major morbidity associated with intraoperative handovers between anaesthetists in cardiac surgery. Similar effects were observed by Saager et al.[2] in general surgery and by Hyder et al.[3] during colorectal surgery.

In recent years, there has been an increasing focus by the Joint Commission in the United States of America on improving handovers.[4] Handover checklists seem to be an easy way to standardise oral communication and to reduce loss of information. Several well designed studies have shown their positive effect on postoperative handovers by anaesthetist to postanaesthesia care unit (PACU) nurses.[5-7]

Although the intraoperative period is equally critical, there have been relatively few studies on transfer of care in the operating room. Most of these studies are difficult to interpret due to methodological bias stemming from the fact that the observations were not blinded.[8-13] In addition, the effect of handover protocols and checklists may be difficult to extrapolate to other countries, particularly when standardised communication is not yet a common part of everyday practice (currently the situation in France).

We hypothesised that the implementation of intraoperative handover training and display of a checklist would improve communication during anaesthesia care transition in the operating room by increasing the number of items of information transmitted.

Methods

The study received Research Ethics Board (REB) approval by the REB in Toulouse (no. 10-0116, dated 11 February 2016, Chair: Dr Nathalie Nasr, CHU Toulouse, Place du Docteur Baylac, Toulouse 31000 France, including a waiver for written consent, because the study was anonymous). All data recording was anonymous with regard to patient and anaesthesia team members.

In preparation for the study, and following a review of the relevant literature, a checklist of 22 handover items was created and validated by a modified DELPHI method among anaesthesia residents and staff (a total of three iterations until consensus among 23 individuals).[6,8]

The interventional cohort study was carried out from January to May 2016 at the University Hospital in Toulouse (CHU Toulouse), which has two different hospital sites in the same city. Intraoperative handovers among residents and nurse anaesthetists were first evaluated simultaneously at the two sites over a period of 2 weeks at ‘baseline’. Following this initial evaluation, one site served as the ‘intervention’ site, in which an intraoperative handover training and display of the laminated 22-item checklist (Table 1) at each anaesthesia workstation were implemented; the other site acted as the ‘control’ site, with no intervention. There were two further evaluations at both sites which followed the ‘baseline’ evaluation: ‘immediate’, over a period of 2 weeks just after training and display of the checklist; and at ‘3 months’, over a period of 1 week, 3 months after protocol implementation. All observations were blinded, meaning that none of the healthcare providers knew that the handover was being observed and scored. This was accomplished by putting the observers (two nurse anaesthetist students) in the operating room close to the anaesthesia team under a completely different pretext unrelated to the anaesthetic (verifying equipment or observing the surgery). At the end of the study, an anonymous satisfaction survey was carried out among the anaesthesia providers in the intervention site.

Table 1

Content of the handover checklist

Preoperative

Intervention (history of present illness, elective vs. emergency, etc.)

Patient (name, age, sex, ASA status, etc.)

Past history (medical, surgical, anaesthetic, family, etc.)

Allergies

Medications (anticoagulants, antibiotics, cardiac, diabetic, etc.)

Team (anaesthesia, surgeon, nurses, etc.)

Intraoperative

Anaesthetic technique

Induction

Airway – Cormack/Lehane

Equipment (vascular access, monitoring, etc.)

In & Out (iv fluids, blood products, urine output, blood loss, etc.)

Injections (neuromuscular blockers, PONV prophylaxis, narcotics, antibiotics, vasopressors, etc.)

Remaining time

Haemodynamic and respiratory stability

Postoperative

PACU (plan for emergence: extubation/intubation; lab, radiograph, regional, pain, etc.)

Postoperative orders

Disposition (ambulatory, floor, monitored bed, ICU, etc.)

Miscellaneous

Special clinical/study protocol

OR planning (following case, set-up and materials, etc.)

Documentation of transfer of care

ASA, American Society of Anesthesiologists’ physical status; OR, operating room; PACU, postanaesthesia care unit; PONV, postoperative nausea and vomiting.

The handover training session consisted of two 1-h meetings and was offered at the intervention site after the first observation phase (‘baseline’) had been carried out, The sessions included a brief discussion of the literature and an introduction to the laminated checklist. Several mock handovers were then practised using simulated cases. No information was given about scoring of the handovers or about the study to avoid an observer effect. After the handover training sessions, the laminated checklists were permanently displayed in each operating room.

Only handovers made inside the operating room (by nurse anaesthetists and residents) were scored. The handovers between attending anaesthetists could not be observed because they usually occur outside the operating room at unpredictable times and locations. Handovers between (or in the presence of) student nurse anaesthetists were also excluded because of the possible influence of the educational setting between student and teacher. The study duration was limited to 6 months (the length of the residents’ assignment period). There was no crossover of personnel between the two study sites.

A checklist scoring tool was created, which added the individual score for each of the 22 items as follows: −1, if there was an error in communicating an item of information (e.g. stating a wrong allergy or wrong comorbidity); 0, if an item of information was unreported; 0.5, if an item of information was partly communicated; 1, if an item of information was communicated correctly (e.g. correctly stating all the patient's allergies or comorbidities). The decision about which components were reasonably expected to be transmitted in the case of several components constituting one item (e.g. multiple allergies or comorbidities) was left to the discretion of the trained observer and generally aligned with documentation on the patient record. The maximum score possible was thus +22 points.

The data collection was done by two student nurse anaesthetist evaluators using their smartphones. Study data were collected and managed using Research Electronic Data Capture tools hosted at Digital Life/Paul Sabatier University.[9] The two evaluators had completed training using simulated handovers and confirmed 100% inter-rater agreement during a trial live observation in an operating room at a site that was not involved in the study.

The primary outcome was the checklist score. Secondary outcomes were handover duration, handover documentation in the anaesthesia record, and satisfaction with the checklist and the handover training session. The study was planned such that a sample of 40 at baseline and 20 at follow-up would have 90% power at α = 0.05 to detect a difference in mean scores of 2.7 or greater, if SD of the score was 3.

All outcomes were assessed by site and time period. Mean, SD and 95% confidence interval (CI) for means were calculated for checklist scores. Differences between means for sites, times and their interaction were studied with analysis of variance (ANOVA), employing Tukey's Honestly Significant Differences (HSD) test for post-hoc analysis. Median and inter-quartile range (IQR) were calculated for handover duration. Differences were compared with the Kruskal–Wallis rank sums test, using Dunn's tests for post-hoc analysis. The proportion of handovers accompanied by documentation was calculated, and differences were evaluated using Fisher's exact test. All P values were two-sided and statistical significance was determined using P less than 0.05. All calculations were undertaken using R statistical software package V3.2.5 (R Core Team (2016). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria).

Results

A total of 204 handovers were observed over a 5-month period – 86 in the intervention site and 118 in the control site. Checklist scores at ‘baseline’ were similar with mean (SD) values of 7.6 (2.6) and 8.8 (3.2) for intervention and control site respectively, increasing to 10.9 (3.8) and 11.0 (2.2) for the intervention site immediately after intervention and 3 months after intervention, respectively; at the control site, the scores at these times remained similar to ‘baseline’ at 6.7 (2.6) and 6.6 (2.6). Figure 1 shows mean (SD) scores by site and time. ANOVA showed a significant site by time interaction effect (P < 0.001). Post-hoc testing with Tukey's HSD showed significant increases in mean (95% CI) scores in the intervention group of 3.4 (1.3 to 5.4) immediately after and 3.4 (1.0 to 5.8) 3 months later (both P < 0.001). The control site showed decreases from ‘baseline’ of −2.1 (−3.7 to −0.4) (P = 0.007) and −2.2 (−4.4 to 0.1) (P = 0.064) for the corresponding time periods.

Fig. 1

Intraoperative handover quality score. See text for details. Mean values. Bars represent SD. ∗P < 0.001 compared with baseline, and between sites.

Durations of handovers are shown in Table 2. The Kruskal–Wallis test, which included site, time and their interaction, indicated a significant difference between sites. Dunn's post-hoc test was used for pairwise comparisons. At the intervention site, the duration of handovers did not increase after handover training and display of the checklist. Median (IQR) duration at ‘baseline’ was 2 (1 to 3) min; immediately after intervention, the duration was 1 (1 to 1.5) min, and 3 months later, it was 2 (2 to 3) min. Site comparison at matching times showed shorter durations of handover at the intervention site at ‘baseline’ and immediately postintervention. At 3 months, the duration was similar at both sites.

Table 2

Duration (min) of intraoperative handovers

	Control site		Intervention site
	Median (IQR)	Comparison with baseline P	Median (IQR)	Comparison with baseline P	Site comparison P
Baseline	3 (2 to 4) (n = 47)	–	2 (1 to 3) (n = 42)	–	<0.001
Immediate	3 (2 to 3) (n = 51)	0.66	1 (1 to 1.5) (n = 27)	0.62	<0.001
3 months	2 (2 to 3.25) (n = 20)	0.63	2 (2 to 3) (n = 17)	0.10	0.77

IQR, inter-quartile range.

At ‘baseline’, there was very poor documentation of handovers at both sites (Table 3). The handover training and display of the checklist improved the documentation of handovers significantly at the intervention site.

Table 3

Documentation of handover

	Control site	Intervention site	Site comparison P
Baseline	3/47 (6.3%)	4/42 (9.5%)	0.70
Immediate	0/51 (0%)	13/27 (48.1%)	<0.001
3 months	2/20 (10%)	7/17 (41.2%)	<0.001

Data are number (%) of instances in which the handover was documented.

The response rate of the satisfaction survey was 88%. Satisfaction with the laminated checklist was 68%, and 77% with the handover training sessions. Just over a quarter of the respondents reported that they had already used another handover checklist before the study started (although no checklist use was observed during the handovers at ‘baseline’) and 32% indicated that they used the checklist in at least three out of four cases after the training. Free comments from respondents indicated that obstacles to using the checklist in every case were as follows: a discrepancy between the organisation of the items (according to an acronym) on the checklist compared with the usual mental model when giving a report; the perception that the use of the checklist was time-consuming; and the opinion that such a detailed handover with 22 different items may not be necessary for healthy patients undergoing low-risk surgery.

Discussion

The implementation of our intraoperative handover protocol (teaching sessions and display of checklists in each operating room) improved the communication of items during anaesthesia handovers in the operating room by 43%. This improvement persisted at 3 months and was not associated with increased handover duration. Handovers after protocol implementation were documented four times more often when compared with ‘baseline’.

The use of checklists remains relatively infrequent, and the low scores emphasise the need to improve communication during transfers of care. In our study, less than 30% of the anaesthesia team members indicated that they had already used a handover checklist prior to the start of our study. In the United States of America, only half of the anaesthesia teams used a standardised handover, and the majority of those felt that the process was insufficient.[10]

Agarwala et al.[11] showed poor handover quality at baseline and were able to improve the communication of a number of important items significantly after implementation of a checklist (e.g. vasopressors from 44 to 85%, antibiotics from 63 to 97%, postoperative plans from 43 to 92%).[11] Likewise, Boat and Spaeth[12] found that a paediatric handover checklist significantly improved the quality of patient handoff in the operating room (from 20 to 100% at 3 months), and Starmer et al.[13] also observed an increase in the number of complete handovers as well as a decrease in errors during handovers made by paediatric residents using their checklist bundle (Illness severity, Patient summary, Action list, Situation awareness and contingency plans and Synthesis by receiver).

Although the efficacy of various protocols and checklists for the transfer of care seems to be confirmed by our study, there is often a concern about possibly increasing the time required for handovers. However, in common with previous findings,[11,13] our study showed handover times of 2 to 3 min which were not prolonged by the implementation of the handover training and display of the checklist. Because a significant number of healthcare providers in our study were concerned about the time required for using the handover checklists, and since this may have been one of the factors explaining why not everybody adopted the protocol, it is important to share this evidence with staff in the hope that this will increase utilisation. The literature[11,12,14-16] shows that successfully implementing a checklist protocol can improve satisfaction, and we found the same.

Although Boat and Spaeth[12] showed 100% participation 2 months after their checklist implementation, Agarwala et al.[11] checklist use was only 60% at 3 months and 74% at 8 months. Changing professional practice is difficult and takes significant time and effort; in our study, only 30% of the anaesthesia providers said they had fully integrated the checklist into their daily practice at 3 months. Multiple reasons were listed as potential obstacles: providers felt that the checklist design did not correspond with their mental model of giving a report, that they were not part of the project's design and implementation, and that using the checklist was time-consuming and too complicated for healthy patients having minor surgery. Also, due to scheduling conflicts, some anaesthesia providers could not attend the educational sessions, and this may have contributed to the low use of the checklist. Based on our experience, we believe that the following elements are particularly important to ensure long-term success: a local leader for the project; support by administration; initial consultation with and involvement from stakeholders; adaptation of the module's design to fit local practice; on-going efforts and follow-up to provide feedback and revisions (as necessary); and a strategy to constructively deal with obstructionists. It may also be helpful to incorporate requirements for quality improvement (including training for a structured handover process) into practice guidelines and educational curricula by national societies.

At the end of the study, the anaesthesia providers were given detailed feedback to show that the protocol and checklist use were not associated with a longer handover time and that the lack of their involvement in the protocol planning was due to methodological considerations (the study was blinded to avoid an observer effect).[17] It was also discussed that there is currently no study that shows that a handover training and display of checklists improve communication only during ‘higher risk’ settings, and it was emphasised that communication should be uniformly optimised for all transfer of care settings (i.e. including American Society of Anesthesiologists’ physical status 1 patients having a minor procedure). To address the comments about complexity and mental organisation of the checklist, all providers were invited to participate in a checklist redesign that finally led to the currently used, second generation checklist. This new version has been used to create a smartphone application (available for free download for iOS and Android – ‘ANESLIST’) that visually represents all items in the form of an interactive checklist.

Our study rated the handover checklist score only with a global score (reflecting the number of items transmitted) without evaluating the details of the healthcare provider communication and was not formally validated. Consequently, this approach, coupled with the fact that our study was blinded, did not allow a detailed evaluation of other attributes of handover quality (e.g. introduction, handover comprehension, conflict resolution, etc.), such as used in the study of postoperative handovers by Weinger et al.,[5] or retention of information by the receiver of the communication, as in the study of Agarwala et al.[11]

Although the effect of our handover training and display of the checklist seems clear, the study's design unfortunately does not allow separation of how much of the improvement was due to the training sessions and how much was from the presence of the checklist displayed in each operating room. If one looks at cognitive aids (and checklists) as a form of medical equipment, then it will become clear that optimal benefit can only be achieved through familiarity, and that the educational process will undoubtedly enhance any benefit from its implementation. This process will require long-term engagement of the team leaders to ensure sustained change (owners and early adopters in Boat and Spaeth's words)[12] to initiate and maintain professional practices change before it can ultimately enhance patient safety.

Although these studies show that training and use of a checklist can improve the number of items communicated during handovers, the impact on quality of communication is difficult to quantify, and its impacts on morbidity and mortality remain to be determined in a prospective study. Our study did not evaluate the effect of the handover training and display of the checklist on patient morbidity and mortality because too many patients would have been required to perform a study with sufficient statistical power. Hudson et al. found an important link between intraoperative handovers during cardiac surgery and in-hospital mortality, and a similar association was found among patients undergoing other forms of surgery.[1-3] However, a recent study by Terekhov et al.[18] did not find that the number of anaesthesia handovers was associated with increased odds of postoperative mortality and serious complications. To date, all studies on intraoperative anaesthesia handovers and morbidity/mortality have analysed only retrospective data and are thus subject to significant methodological limitations, such as bias resulting from confounding factors (e.g. the study by Terekhov et al.[18] was performed at an institution where a detailed PACU handover protocol with teaching sessions had just been implemented).

Conclusion

A handover training and display of printed handover checklists in each operating room improved the handover checklist score for the transfer of care in the operating room during anaesthesia. The improvement persisted for 3 months and was not associated with an increased duration of handover. Checklists can be effective communication tools, and their availability in the form of smartphone applications may further increase their adoption by healthcare providers to benefit patient safety.