EMR-based handoff tool improves completeness of internal medicine residents' handoffs.

BACKGROUND: The majority of adverse events in healthcare involve communication breakdown. Physician-to-physician handoffs are particularly prone to communication errors, yet have been shown to be more complete when systematised according to a standardised bundle. Interventions that improve thoroughness of handoffs have not been widely studied. AIM: To measure the effect of an electronic medical record (EMR)-based handoff tool on handoff completeness. INTERVENTION: This EMR-based handoff tool included a radio button prompting users to classify patients as stable, a 'watcher' or unstable. It automatically pulled in EMR data on the patient's 24-hour vitals, common lab tests and code status. Finally, it provided text boxes labelled 'Active Issues', 'Action List (To-Dos)' and 'If/Then' to fill in. IMPLEMENTATION AND EVALUATION: Written handoffs from general and specialty (haematology, oncology, cardiology) Internal Medicine resident-run inpatient wards were evaluated on a randomly chosen representative sample of days in April and May 2015 at Stanford University Medical Center, focusing on a predefined set of content elements. The intervention was then implemented in June 2015 with postintervention data collected in an identical fashion in August to September 2016.
RESULTS: Handoff completeness improved significantly (p<0.0001). Improvement in inclusion of illness severity was notable for its magnitude and its importance in establishing a consistent mental model of a patient. Elements that automatically pulled in data and those prompting users to actively fill in data both improved.
CONCLUSION: A simple EMR-based handoff tool providing a mix of frameworks for completion and automatic pull-in of objective data improved handoff completeness. This suggests that EMR-based interventions may be effective at improving handoffs, possibly leading to fewer medical errors and better patient care.

Introduction

Problem description

Communication breakdown plays a part in the majority of adverse events in healthcare.1 In particular, transitions of care, including handoffs from one physician team to another, are frequently associated with communication errors and thereby with adverse events.2 3 While AmericanAccreditation Council for Graduate Medical Education (ACGME)-instituted resident work hour maximums are meant to improve patient safety by decreasing provider fatigue, such improvements may be offset by the increased handoffs that necessarily result from shorter shifts.

Handoffs represent a particularly error-prone time, as incomplete or even incorrect information is often communicated. Indeed, in one study, the most important piece of information was not communicated in 60% of intern handoffs.4 A more systematic handoff process is necessary to prevent these communication lapses and their associated errors.

Available knowledge

The gold standard system for this handoff process is the I-PASS paradigm.5 Thus, many recent efforts to improve patient handoff safety have focused on improving I-PASS implementation, particularly concerning written handoff documents. Recent additions to the paediatric literature have shown that by implementation of a ‘handoff bundle’ of interventions, it is indeed possible to increase precision and completeness of both written and verbal handoffs and thereby significantly decrease medical errors.6 7 These bundles have included electronic medical record (EMR)-embedded support for written handoffs, as well as a number of other interventions. Other authors have examined the effects of EMR-based handoff tools on physician-reported handoff quality and workflows and found favourable effects.8–11

Rationale and specific aims

To our knowledge and based on recent reviews,12 13 there are limited data specifically evaluating these EMR tools on objective measures of handoff quality and/or patient outcomes in the adult population. Our quality improvement (QI) study addresses this knowledge gap via analysis of a new EMR tool for written handoffs, with the hypothesis that implementation of an EMR-based handoff tool would improve adhesion to the I-PASS handoff system.

Methods

Context

Study design

We conducted a prospective systems-based intervention study on adult medicine inpatient units at a major academic medical centre in the USA after receiving Institutional Review Board exemption for this QI study. Baseline data were collected in April and May 2015, with implementation of the intervention in June 2015 and postintervention data collected in August and September 2016. Data on the quality of written handoffs were collected for all patients on the study units on randomly chosen days during the data collection period.

Study team descriptions

The study teams included five general medicine teams and specialty medicine teams covering cardiology, oncology and haematology patients during the day, each composed of approximately two interns and one resident at a time, as well as two interns and several residents covering all these teams’ patients at night. Non-call daytime interns typically signed out to on-call daytime interns by 17:00, who then signed out all the teams’ patients to the covering night intern. Work schedules were the same before and after the intervention.

Handoff in the preintervention period

Before the intervention, written handoff notes were updated daily by interns and/or residents using a template in the EMR, Epic Systems Hyperspace 2015 (Epic, Verona, WI). Other providers could not access this template. Verbal handoff was intern to intern and resident to resident. Residents were not routinely present for intern handoffs or vice versa. While house staff had previously been educated about principles of high-quality handoffs, the written template was entirely free text without prompting of any particular element. Only basic patient demographic information, including name, medical record number, attending physician and code status, was pulled into the preintervention printed written handoff document.

Intervention

The study intervention was the implementation of an Epic-based handoff template based on I-PASS handoff elements. The template replaced the free-text handoff note described above and included separate free-text boxes for a patient summary, to-do list, and contingency planning as well as a radio button drop-down menu prompting the inclusion of an illness severity evaluation. The new template was developed through focus groups with Internal Medicine and Surgery residents as well as a multidisciplinary group that included hospitalist and nursing leadership. When printed, the tool automatically pulled in information on updated patient medications, allergies, vital signs and basic laboratory test results without any action on the part of the user in addition to demographic information already included on the earlier version (which included updated code status, as above). The computer-based template before and after completion is pictured in figures 1 and 2, respectively; the printed handoff is pictured in figure 3.

Figure 3

Printed handoff (©2017 Epic Systems. Used with permission).

On-screen handoff tool prior to addition of content (©2017 Epic Systems. Used with permission).

On-screen handoff tool after addition of content (©2017 Epic Systems. Used with permission).

The handoff tool was implemented in Epic for all services at once and placed in the ‘landing page’ for all providers and nurses to increase visibility to all disciplines. During implementation, education involved presenting the new Epic handoff tool at Internal Medicine noon conferences and intern orientation. Use was not explicitly enforced after implementation.

Measures and analysis

We assessed rates of inclusion of I-PASS-recommended handoff elements in a random sample of written handoffs. Every written handoff document was printed three times weekly, and each written handoff document on four randomly chosen days was assessed by a single investigator. Following previously validated methods and by consensus of study coinvestigators,6 7 each was assessed for the presence or absence of nine I-PASS elements. These elements and criteria are listed in table 1. Written handoffs were coded as same-day admissions or not, and same-day admission handoffs were removed from the data set if they were missing patient summary, to-do list and contingency plans as this implied the handoff had been printed by the study team before the admitting team had prepared to sign out.

Table 1

Handoff elements assessed

Element	Criteria for inclusion
Illness severity assessment	‘Stable’, ‘unstable’ or ‘watcher’ appeared anywhere in handoff ‘Hemodynamically stable’ not sufficient
Patient summary	At least three of: summary statement, events leading up to admission, hospital course, ongoing assessment and active plans
To-do list	Clearly written list of items to do or ‘nothing to do’ or ‘NTD’
Contingency plans	At least one indication of what to do if adverse contingencies occurred or clear statement that no adverse contingencies anticipated
Allergies	Clear statement of allergy
Code status	Any indication of code status other than ‘prior’, that is, from past admission
Meds	At least one clearly current medication listed, dose not necessarily included
Labs	At least one clearly recent laboratory result. Qualitative results (eg, ‘hyperkalemia’) were included
Vital signs	At least one clearly recent (ie, from day of handoff) vital sign. Qualitative results (eg, ‘tachycardic’) were not included. Quantitative ranges (eg, ‘120–130’) were included

We additionally surveyed nurses on medicine units about their perceptions regarding the utility of the medical doctor (MD) handoff, as they had requested more transparency in the physician handoff report and the tool was designed for nurses to access. This survey used a 5-point Likert scale to assess whether nurses used the handoff report to understand the plan of care for the patient, whether they found the information therein helpful, and the impact of this information on the frequency of paging MDs overnight.

Results

A total of 288 written handoff documents were assessed during the preimplementation study period in April and May 2015, representing 241 unique patients. Sixty-one per cent of these written handoffs corresponded to general medicine service patients, with the remaining 39% corresponding to patients being cared for by haematology, oncology, or cardiology services. The intervention was implemented in June 2015. A total of 166 written handoffs were assessed during the postimplementation study period in August and September 2016, representing 144 distinct patients. There was no difference in completeness for any content element between general medicine and specialty service patients in either study period (online supplementary appendix I). The proportion complete for a given element was compared by Χ2 test between all preintervention handoff samples and all postintervention handoff samples, as well as between general medicine and specialty service completeness for each element. Statistical analyses were performed using MedCalc (MedCalc Software, Ostend, Belgium, medcalc.org).

As shown in table 2 and figure 4, the proportion of handoffs that included medications, laboratory tests, vital signs and a clear assessment of illness severity increased significantly between the preimplementation and postimplementation periods. These former three elements were pulled in automatically by the tool, whereas illness severity was filled out actively. Those elements that were relatively complete prior to the tool implementation, that is, patient summary, code status, to-do list and contingency, remained so after implementation. The element for patient allergies essentially served as a negative control, being rarely included in either preimplementation or postimplementation handoffs.

Table 2

Written handoff completeness, preintervention and post-intervention

	% of handoffs complete Preintervention	% of handoffs complete Postintervention	∆ (P values)
Patient summary	100.0	100.0	0.0% (1.0)
Code status	100.0	100.0	0.0% (1.0)
To-do list	97.6	95.8	−1.8% (0.2839)
Contingency	95.8	98.8	3.0% (0.0762)
Meds	93.8	100.0	6.2% (0.0011)
Labs	36.5	98.8	62.3% (<0.0001)
Vitals	18.4	100.0	81.6% (<0.0001)
Illness severity assessment	1.0	98.8	97.8% (<0.0001)
Allergies	0.3	0.0	−0.3% (0.8236)

Values in bold connote statistically significant differences between pre- and post-implementation values.

Figure 4

Written handoff completeness, pre- and post-intervention.

Regarding the nursing surveys on handoff information, 33/35 nurses surveyed (94%) answered ‘yes’ or ‘sometimes’ when asked whether they used the MD handoff report to understand the plan of care for the patient; 28 of those 33 (85%) reported finding the information contained in the handoff ‘very helpful’ or ‘somewhat helpful’; and 19 of those 33 (58%) reported that the information in the handoff decreased the frequency with which they paged the on-call MD team overnight.

Discussion

Summary

This simple EMR-embedded tool was associated with large increases in written handoff completeness both for data that are automatically pulled in by the handoff tool and for data that must be filled in by users, even more than a year after implementation. Notably, in this latter category, the proportion of handoffs including an assessment of illness severity increased dramatically. The overwhelming majority of nurses surveyed reported utilising the handoff information and finding it helpful, and more than half of those utilising the handoff information reported that it decreased the frequency of MD-directed pages overnight.

Interpretation

To our knowledge, we are among the first to show that implementation of an EMR-embedded tool can be associated with objective increases in written handoff completeness. Our method of scoring for inclusion likely underestimates true changes in completeness, as preimplementation medications included only a subset of those administered for a given patient whereas all medications ordered were pulled into the handoff postimplementation. The same was true for laboratory tests and vitals. One exception may be illness severity assessment, as given the very low rate of inclusion preimplementation we may have defined this parameter overly narrowly; still, the very high rate of inclusion postimplementation suggests that the template was likely successful in prompting users to include it. Given the importance of illness severity assessment for forming and communicating a concise shared mental model of a patient’s clinical status, this is a key finding.

To the extent the EMR tool was responsible for these changes in handoff practices, one can posit potential mechanisms for the change. In general, well-designed EMR tools can significantly decrease the cognitive energy required to navigate the vast array of information with which physicians constantly contend, and can absorb much of the information that would otherwise consume the physician’s working memory. In this particular case, the EMR tool also improved the availability of a shared mental model of each patient’s clinical status to multiple physician and non-physician providers. Based on prior research suggesting that EMR-based handoff information is used by a variety of clinician cadres11 and the nursing survey data above, it is likely that making this information explicit and accessible in the EMR has spillover effects beyond the direct handoff process. Based on this survey, this included decreasing the frequency of night-time pages, thereby minimising interruptions to MD workflow, though quantitative paging data to support or refute this subjective perception were unfortunately not available.

Notably, the EMR tool was essentially a bundled intervention in itself in that it involved prompts to include several I-PASS elements as well as workflow changes, such as the ability to write the handoff while simultaneously viewing other data for that patient. It is therefore difficult to precisely identify which feature(s) of the tool was/were most closely associated with more comprehensive written handoffs. However, prior work has introduced an EMR-based handoff as part of an even larger intervention bundle; hence, our approach is already narrower than that of existing literature.

Limitations

Previous work shows convincingly that as part of a bundle of interventions, improved handoffs via more complete implementation of the I-PASS protocol can translate to better patient care with fewer adverse events.6 However, our study was limited in that it did not itself evaluate patient safety or outcomes, or indeed whether the increased completeness of the postintervention handoffs truly represented an improvement in handoff quality.

Increased completeness carries two major risks: first, the presence of conflicting information, and second, possible information overload. Regarding the former, the scope of our study did not include verification of the accuracy of the free-text handoff information or the extent to which it was updated. However, anecdotally, many postintervention handoffs did include the most pertinent medications, laboratory tests and vital signs in the free-text parts of the handoff in addition to the automatically generated parts, and at times these free-text values were at odds with those pulled in automatically. These discrepancies likely reflect a failure to fully update the written handoff daily for patients with longer lengths of stay, and can be interpreted as supportive of automating at least a subset of handoff data based on the patient’s current orders to guarantee that information is fully updated. However, the presence of blatantly conflicting information also highlights the potential for confusion on the part of covering providers—particularly if the error is not noted at the time of sign-out, when it can prompt verbal discussion and clarification. Such conflicting information also dilutes the aforementioned positive spillover effects for other providers. A manual audit to evaluate the accuracy of free-text handoff information and the frequency of contradictory information is a key investigation going forward.

Regarding the latter, increased completeness via automatically pulling in more information may have also resulted in the inclusion of extraneous information and information overload for providers. Surveying residents about the ease of locating relevant information and perception of increased data as useful or distracting would have been useful, and future work should include such a survey, although without a preimplementation comparator—alas, the timing of the tool’s implementation at the change in an academic year complicated collection of this information at the time of the original study.

Finally, another major limitation of this study was the observational nature of tool implementation; given that it was implemented across all study teams simultaneously versus in a randomised fashion and over multiple classes of trainees, one cannot definitively attribute the changes in handoffs to this change.

Conclusions

In sum, our study adds to the limited literature evaluating EMR-based handoff tools by demonstrating that in the adult population, a relatively simple tool can significantly increase handoff completion as operationalised by I-PASS implementation. This improvement extended to data that users were required to actively enter, most significantly an assessment of patient illness severity—a critical piece of information for promotion of shared mental models of clinical status among providers. Nurses also reported that placement of the handoff tool in a visible section of the EMR improved their knowledge of the plan of care for the patient and decreased paging to overnight teams. Yet, we note that completeness is only one of several important determinants of a quality handoff; much more work is needed to evaluate handoff efficiency—that is, the communication of all necessary information without extraneous data or information overload—and free-text handoff accuracy. Indeed, the finding that such a simple intervention can so change handoff information ought to be viewed with caution, given these unintended consequences. Future work must explicitly investigate the effects of these changes on patient safety and medical errors to ensure that such templates do in fact improve care.