Improving quality of care in less than 1 min: a prospective intervention study on postoperative handovers to the ICU/PACU.

PURPOSE: Standardisation of the postoperative handover process via checklists, trainings or procedural changes has shown to be effective in reducing information loss. The clinical friction of implementing these measures has received little attention. We developed and evaluated a visual aid (VA) and >1 min in situ training intervention to improve the quality of postoperative handovers to the intensive care unit (ICU) and postoperative care unit.
MATERIALS AND METHODS: The VA was constructed and implemented via a brief (<1 min) training of anaesthesiologic staff during the operation. Ease of implementation was measured by amount of information transferred, handover duration and handover structure. 50 handovers were audio recorded before intervention and 50 after intervention. External validity was evaluated by blinded assessment of the recordings by experienced anaesthesiologists (n=10) on 10-point scales.
RESULTS: The brief intervention resulted in increased information transfer (9.0-14.8 items, t(98)=7.44, p<0.0001, Cohen's d=1.59) and increased handover duration (81.3-192.8 s, t(98)=6.642, p=0.013, Cohen's d=1.33) with no loss in structure (1.60-1.56, t(98)=0.173, p=0.43). Blinded assessment on 10-point scales by experienced anaesthesiologists showed improved overall handover quality from 7.1 to 7.8 (t(98)=1.89, p=0.031, Cohen's d=0.21) and improved completeness of information (t(98)=2.42, p=0.009, Cohen's d=0.28) from 7.3 to 8.3.
CONCLUSIONS: An intervention consisting of a simple VA and <1 min instructions significantly increased overall quality and amount of information transferred during ICU/postanaesthetic care unit handovers. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Data collected by The Joint Commission demonstrate the critical role of communication in the generation of sentinel events. From 1995 to 2005, two-thirds of all sentinel events were traced back to ineffective communication1 and information loss consistently ranks in the top three root causes of sentinel events.2 Handovers are particularly vulnerable to such information loss3 and communication failures during handovers have been implicated in up to 80% of adverse events.2 While clinical monitoring and information systems have the purpose of allowing easy retrieval of information that is actively searched for ("pull"-communication), patient handovers present one of the few situations in which patient information is actively passed from one caregiver to another ("push"-communication).

Simple visual aids (VA; eg, checklists or proforma prompts) have been identified as one suitable tool for quality improvement in healthcare.4 5 A prospective intervention study on the introduction of the WHO Surgical Safety Checklist in eight hospitals with diverse populations and economic circumstances, for example, has shown a marked and significant reduction in postoperative mortality and complication rates.6 In the wake of these promising results, several checklists and proformas have been developed for various procedures, among them postoperative handovers.6–8 Studies on checklist effectiveness in improving patient-specific health outcomes have provided mixed results, however, and it is assumed that much of the variation is due to differing degrees of effective implementation.9 10

Systematic reviews on checklist implementation have identified the following factors to be important to their effective utilisation: engagement of stakeholders and human factors experts in the design; development based on end user needs and realities; grouping sections by task or chronological order; pilot testing and validation before implementation, and continuous updating.11 The Joint Commission has recently developed the SHARE guideline for structured improvement of the handover process as a whole, which posits that a successful and sustainable improvement of handovers must include the following aspects: Standardisation of critical content; Hardwiring of tools in the hospital system; Allowing asking questions; Reinforcement through integration into clinical audits; and Education of staff. Adhering to all the SHARE components, however, is a resource-intensive and potentially disruptive undertaking, specifically with regard to hardwiring, reinforcement by integration into clinical audits and staff education. In addition, the SHARE framework is itself not evidence based and it is yet unclear in how far adherence to the SHARE guidelines is associated with success of intervention. A recent systematic review on interventions to improve handovers12 found no link between compliance to the SHARE guideline and improved outcomes, that is, checklists and proformas improved information retention6–8 13–18 or end user satisfaction7 8 14 15 regardless of adherence to other SHARE dimensions.

One aim of this study was to develop a mnemonic VA that would be as easy and cost-effective to implement and assess as possible, with only a minimal amount of in situ staff training and no structural changes, information technological requirements or administrative anchoring required. In work environments with a high density of procedures, such as intensive care medicine, it is important that any interventions to improve patient safety disrupt existing workflows as little as possible. Our aim was thus to design an end user-friendly and widely applicable tool that would support the development of a handover culture and evaluate its impact on information loss, handover structure, handover duration and overall handover quality.

Materials and methods

Participation in the study was anonymous and voluntary. Neither patients nor the public were involved in the research at any stage of the process.

VA construction

The VA was constructed by an experienced human factors psychologist and anaesthesiologic quality improvement consultant using established methods from human factors psychology and naturalistic decision-making.19 20 The first phase consisted of a literature review followed by an initial conception of a list of critical items to be included in handovers by three expert anaesthesiologists (all 10+ years’ clinical experience since specialist exam) in alignment with already established clinical guidelines and procedures (eg, WHO Surgical Safety Checklist). In the second phase, three further expert anaesthesiologists were each asked to construct a short/standard and long/complex handover script based on this initial VA. The scripts were then presented to intensive care unit (ICU)/postanaesthetic care unit (PACU medical and nursing staff (n=20) for evaluation. Staff were asked to identify both missing and possibly superfluous information, as well as provide input on preferences in structure and layout of the VA and level of detail of the included items. Some items were dropped and the ‘Operating Room (OR)-Personnel’ category included as a result. The ‘OR-Personnel’ section was seen as particularly important to the receiving staff for better accessibility in case of a later need to contact operating staff directly. The resultant list contained 25 specific items and a further five items labelled ‘other’ to mark special cases that would not be applicable to most handovers for a total of 30 items. In the third phase of VA construction, the items were grouped into a coherent chronological and semantic framework containing seven categories and an acronym was devised for mnemonic support: IPA-NOVA—Intro, Personnel, Anamnesis, Narcosis, Operation, Volume Management, Next Actions (see figure 1). Finally, the VA was then sent to all senior anaesthesiologists (n=53) of the department for final review and open commentary. No further changes were made, and the final VA was designed as a laminated, double-sided pocket card able to fit into the chest pocket of the hospital scrub.

Figure 1

Final IPA-NOVA visual aid provided to staff for postoperative handovers to the ICU/PACU (translated from German into English by the authors). BIS, bi-spectral index - technology used to monitor depth of anesthesiaI; CU, intensive care unit; DECT, digital enhanced cordless communication - cordless phone used for hospital communication; PONV, post-operative nausea and vomiting.

VA implementation

We then conducted a prospective observational intervention study consisting of (1) recording unaided handovers prior to the intervention, (2) provision of the VA to the OR-anaesthesiologist during the operation and a brief explanation of its content and purpose (<1 min) with the request to use the VA at the subsequent patient handover, and (3) recording the VA-aided handovers after intervention. Staff were encouraged to use the aid to prepare the handover, could refer to the aid during handover and could keep it after handover completion. Note that, in Germany, receivers of ICU/PACU handovers are anaesthesiologists rather than ICU/PACU nurses due to the different medicolegal framework and accountability structures in Germany. Additionally, due to staff rotation between the OR anaesthesiologists and the ICU anaesthesiologists, we expect a certain degree of ‘cross-pollination’ between the staff conducting the handovers and those receiving them.

Audio recordings were sampled continuously during normal operating hours within a 2-week time period. OR-anaesthesiologists caring for patients known to go to the ICU/PACU after the operation were approached in situ and their handovers recorded. We used the VA as scoring device to assess how well participants were able to integrate it into their handover process in situ, given this very brief intervention. The primary endpoint was amount of information transferred as measured by the 30 items included in the VA. Secondary endpoints were (1) duration of handover and (2) handover structure as measured by the number of ‘return-visits’ to already mentioned IPA-NOVA categories, that is, if the OR-anaesthesiologists had communicated some of the items in the ‘Intro’ category, continued onto the subsequent categories, and then later revisited the ‘Intro’ category to mention further items, this would be counted as a ‘return-visit’. The more return visits a handover contained, the worse the handover structure was assessed. Lastly, we also assessed the number of pieces of information per return visit as additional measure for handover structure. This is because with an increasing number of information, more opportunities for return visits exist.

VA assessment

We then assessed in how far introduction of the VA translated into tangible improvements in ICU handover quality as perceived by experienced anaesthesiologists. The recordings were screened for any mention of information that would allow identification of patients, staff or whether the checklist had been used in the handover (which would reveal the condition) and any such information was removed from the recordings. Preintervention and postintervention recordings were then randomised and given to experienced anaesthesiologists (n=10) for blinded assessment (average 5.75 years since specialist examination). The anaesthesiologists rated the recordings on a scale from 1 to 10 on the dimensions: overall handover quality, completeness of information, handover structure and quality of the audio recording. Each anaesthesiologist received 10 recordings (5 before intervention and 5 after intervention) for blinded assessment, for a total of 100 assessments.

Statistical analysis

For a two-sided, independent samples t-test, assuming at least a moderate effect size (Cohen’s d >0.7) and alpha and beta error thresholds of 5%, we computed the necessary sample size to be 45 handovers per group (pre/post). To guard against data loss we collected an additional five handovers per group for a total of 50 handovers per group.

Results

A total of 100 handovers were audio recorded (50 before intervention and 50 after intervention). On only three occasions, anaesthesiologists requested their handover not to be recorded. For statistical analyses, see table 1.

Table 1

Statistical measures of the effects of the intervention

	T-statistic	Cohen’s d	P value	Significant?
VA implementation
Information transferred	7.443	1.59	<0.0001	Yes
Handover duration	6.642	1.33	0.013	Yes
Handover structure	0.173	/	0.43	No
Information per return visit	5.373	1.09	<0.0001	Yes
VA assessment
Overall quality	1.89	0.21	0.031	Yes
Information	2.42	0.28	0.009	Yes
Handover structure	0.49	/	0.31	No

VA, visual aid.

Two raters coded the recorded handovers independently according to the categories and items on the VA. All recorded handovers were included in this analysis (15 inter-rater disagreements were resolved). Utilisation of the VA increased the information passed on during handovers (as measured by the VA) as well as time taken for handovers, without increasing the number of ‘return-visits’ to previously visited categories. All in all, both a higher information density (information per time) as well as a better handover structure (information per return visit) can be observed (see figure 2).

Figure 2

Information transfer on the x-axis (0–30 VA items). The y-axis in the upper graphs shows handover duration (0–600 s) and in the lower graphs handover structure, that is, return visits to already visited categories (0–6) before (left) and after (right) the intervention. The quadrants in each graph represent different ranges of performance desirability (green=desirable, red=undesirable, orange=acceptable). VA, visual aid; HO, handover.

The mean (SD) number of pieces of information transferred during the handover increased from 9.0 items (3.15) before intervention to 14.8 items (4.42) after intervention. At the same time, the mean (SD) duration of the handovers also increased from 81.3 s (65.3) before intervention to 192.8 s (99.2) after intervention. The mean (SD) number of return visits per handover did not significantly change from 1.6 (1.24) before intervention to 1.56 (1.02) after intervention. Consequently, the mean (SD) number of pieces of information per return visit increased significantly from 6.1 pieces of information per return visit (2.47) before intervention to 11.3 pieces of information per return visit (6.32) after intervention.

For evaluation of the external validity, 10 anaesthesiologists with a mean of 5.75 years of clinical experience since their medical specialty examination (min=1.5, max=13, SD=3.84) participated in this study. On 10-point scales, blinded assessment showed significantly higher perceived mean (SD) overall handover quality, with a score of 7.1 (0.53) before intervention and 7.8 (4.1) after intervention. Mean (SD) perceived completeness of information handed over also increased from 7.3 (0.5) before intervention to 8.3 (3.2) after intervention. There was no improvement in perceived mean (SD) handover structure.

Discussion

In this study, we developed a VA for anaesthesiologists to support them in improving postoperative handovers to the ICU/PACU, using methods from human factors psychology and incorporating all senior staff at one of the largest European Departments of Anaesthesiology and Intensive Care Medicine. The VA was easily understood by staff and significantly increased the information transferred during the handover phase after <1 min in situ training. Participants were able to transfer more information without loss of handover structure (ie, more return visits), which indicates good integration of the VA into existing workflows and speaks to its face validity in reflecting a natural handover process. Overall, we see a notable improvement both in absolute information gain and efficiency of postoperative handovers (see figure 2). In only three instances did staff refuse recording of a handover, showing good overall acceptance and reducing the risk of selection bias.

It is encouraging that the very brief training and limited exposure to the VA was sufficient for experienced anaesthesiologists to perceive the postintervention handovers as being of higher quality in blinded assessment. The greatest effect is again found for completeness of information handed over, while overall quality only shows a small effect and no significant effect in perceived differences in the structure of the handover was found. Our study differs from other studies previously investigating subjective perceptions of handover quality3 14 with its focus on ease of implementation and its use of a randomised, blinded assessment.

The more than doubling of handover duration from 80 to 190 s may be considered a drawback and surprising in light of other studies showing no postintervention increases or even reductions in handover duration. These studies, however, uniformly report a much longer handover duration than was the case here prior to the intervention. Karakaya et al,6 for example, showed a reduction in handover duration from 6 to 4 min, and Nagpal et al3 from 8 to 7 min, far longer than both the 1.5 min observed before intervention and 3 min observed after intervention. Second, substantially more information was transferred. Before intervention, participants took on average 81 s for transferring 9.0 pieces of information, that is, 8.9 s per item. With an average of 14.8 items being transferred after intervention, one should expect handover duration to increase to 131.7 s, all else being equal. The remaining 61 s increase in handover duration is likely attributable to the unfamiliarity with the tool—given less than a minute of instructions and learning the aid in situ, participants were still getting used to the VA while simultaneously applying it. It should also be noted that 3 min per ICU/PACU handover is not an excessive amount of time to allocate to this critical moment in the healthcare process, especially in light of the information gain and increase in handover quality. Furthermore, the extra time spent on a more complete handover may well offset the time spent at a later point in the healthcare process required to retrieve missing information.

More generally, the reference frames of 600 s (10 min), 6 return visits and 30 meaningful pieces of information chosen in figure 1 represent a reasonable space within which an ICU/PACU handover might take place and the overall impact of the intervention along these dimensions is encouraging. However, it should be noted that some variation will always be necessary considering individual peculiarities of the handover and patient history. It is therefore important to stress that our VA is not meant as a top-down quality control measure, but to serve as a mnemonic support tool to ease the introduction of a handover culture to a particular ward.

Several limitations of study design should be mentioned. Regarding handover culture, Boyd et al12 highlight the fact that handovers are a team-based activity and the need for a shared mental model. While, in Germany, both those conducting and receiving handovers are anaesthesiologists, one limitation which should be addressed in future studies is the degree to which handover information is retained on the side of the receiving personnel and the effect of training of receiving personnel on such information retention. When both sender and receiver share the same ‘roadmap’ with regard to the structure of handover communication, more information should be retained, and the process will also be less susceptible to information losses as a consequence of interruptions or other forms of clinical friction. Given staff rotations between the OR staff, who received the brief training intervention, and the ICU/PACU staff, who received the handover, we can expect that, particularly towards the end of the study, many anaesthesiologists receiving handovers in the ICU/PACU had already themselves been trained previously. However, the fact that the study neither captured the degree of this ‘cross-pollination’ nor assessed posthandover information retention directly is a clear limitation. Pucher et al13 suggest integration with the electronic medical record and autopopulation of handover information as the ideal of Hardwiring interventions (see SHARE domains in the Introduction section). An important aspect of handovers is, however, that they are one of the few cases of PUSH communication in an increasingly automated healthcare, and especially intensive care, system. In PUSH communication, the actual experience of patient care during the intraoperative phase and idiosyncrasies of individual patients can be transported at far greater resolution than is currently possible with automated systems. Important details that provide critical context at later stages of patient care should continue to be communicated actively and interpersonally. Automatically transferring standardised information and having to later extract it from the information environment (eg, electronic medical record) in the form of PULL communication effectively means that the trajectory of patient care is reconstructed rather than actively passed on from one caregiver to the next.

Additional limitations of the study design are that there was no systematic assessment of the effect of the intervention on the great variety of metrics applicable to ICU/PACU handovers. This includes qualitative metrics such as ‘professionalism’ or ‘ability to communicate’. While a degree of qualitative assessment is contained in the subjective assessment of experienced anaesthesiologists on the metric ‘overall handover quality’, this is a very coarse measure that does not allow for identification of precisely which qualitative aspects of the handovers require improvement or special attention. The study also did not assess the impact of the intervention on other forms of information flow through the organisation such as documentation in the electronic health record. Anecdotally, the staff conducting the assessment did not observe any negative impact of the introduction of the VA on these aspects, but it is important that an intervention and improvement of one aspect of a good ICU/PACU handover (eg, the verbal aspect) does not impede other equally important aspects.

Lastly, the study did not directly address the sustainability of the intervention. This is an important factor, as even a low-cost intervention that is not sustainable is still a bad investment of time and resources. A critical aspect of the sustainability of an intervention is its adaptation to the local setting and continuous assessment. We therefore also aimed to maximise the aspects of the assessment that can be executed ex situ. In this study, evaluation of its effectiveness took place entirely ex situ with no disruption of clinical workflows or extraction of staff for posthandover interviews. Yet, while this should go some way in supporting the continuous assessment and feedback as a necessary requirement for adaptation to the local setting, we neither collected data on the sustainability of the intervention, say, 6 weeks after, nor did we address interventions shown to increase sustainability such as employee trainings or the implementation of monitoring or reporting structures.

A frequent criticism of observational studies is that they are subject to the Hawthorne effect—the finding that people tend to perform better when they are being observed. This effect is mitigated in our study by the predesign/postdesign. The Hawthorne effect would either be present in both situations equally or may even become smaller as ICU/PACU staff become habituated to the presence of the observer over time, meaning that the Hawthorne effect would be stronger in the preintervention handovers, that is, the effect would go in the opposite direction of what we observed. Second, both anecdotally from assessing the recordings as well as when looking at the very short duration of the handovers, any efforts by staff to improve performance simply due to being under observation can only have been marginal at best. Lastly, it is important to stress again that the purpose of our intervention was not the implementation of a top-down quality assurance measure, but a tool to encourage and facilitate becoming better at an aspect of the medical profession that is gaining increasing recognition for its relevance. In a collaborative (participatory) rather than adversarial (top-down hierarchical/quality control) mindset, the Hawthorne effect can play an important role in successful implementation and should be used to the fullest extent possible. In this context, the Hawthorne effect simply means increased depth of processing of the contents of the new tool, which is exactly what is desirable in its implementation.

Conclusion

In conclusion, a short and cost-effective intervention consisting of a simple VA and a <1 min training can significantly improve the information gain during handovers and increase handover quality as assessed by experienced anaesthesiologists in blinded assessment. Future studies will investigate the impact of widespread VA implementation on patient health outcomes and in the long term. In addition, the effects of participation-driven interventions on anchoring structured patient handovers in the ward culture, using longitudinal designs to assess impact, should also be investigated.