Assessment of a Brief Handoff Skills Workshop for Incoming Interns: Do past Handoff Experiences Impact Training Outcomes?

BACKGROUND: Patient care handoffs are a core professional activity that incoming interns are expected to perform without direct supervision upon starting residency, yet training in medical schools is inconsistent.
OBJECTIVE: To implement a brief handoff communication workshop for incoming interns and determine whether learner-level determinants were associated with differences in training outcomes.
METHODS: We conducted a one-hour interactive handoff skills workshop for all incoming interns at a Midwestern academic medical center. We performed paired pre/post-intervention assessments of participants' attitudes and ability to perform representative handoff skills. The results were analyzed in aggregate and based upon participants' prior handoff experiences using Wilcoxon signed-rank test.
RESULTS: Ninety-nine of 108 interns (91.7%) completed both pre- and post-surveys. There was significant improvement in all 10 attitude-based questions (P ≤ 0.014 for all) and on the skills assessment (1.07 vs 2.16 on 0-4 point scale, SD 1.25, P ≤ 0.001). Results remained significant regardless of prior training, number of handoffs observed, number of handoffs performed, medical school, or residency discipline.
CONCLUSION: A brief interactive workshop for incoming interns can improve participants' confidence and performance of basic handoff skills, regardless of previous training or experience.

Introduction

Handoffs are exchanges of information between health providers that accompany transfer of patient care responsibility and accountability. Poor handoff communication is associated with adverse patient outcomes,[2-6] and in the past decade various oversight organizations have made handoffs a priority.[7-9] Resident work hour restrictions have increased the number of handoffs at teaching hospitals, and the Accreditation Council for Graduate Medical Education (ACGME) requires that sponsoring institutions and programs ensure structured and effective handoff communication.

More recently, the Association of American Medical Colleges (AAMC) identified handoffs as a core entrustable activity for graduating medical students. Although handoff training interventions for medical students have been described,[13-16] few medical schools have implemented handoff curricula. Even when such training occurs, there is some evidence that medical students may not retain handoff skills in the following months, supporting the need for subsequent education prior to beginning residency. A recent study showed that a day-long handoff workshop for incoming interns could improve self-reported perceptions, but time constraints may limit its applicability, and the influence of participants' prior handoff experiences is unclear. With this in mind, we implemented a brief handoff communication workshop for incoming interns and examined whether learner-level determinants were associated with differences in training outcomes.

Methods

Setting and participants

We conducted the intervention in June 2014 at a large, state-funded academic medical center in the Midwest. Under the direction of the associate dean for graduate medical education, a multidisciplinary team of physicians from internal medicine, family medicine, emergency medicine, general surgery, and anesthesiology developed the handoff skills workshop. The one-hour training was targeted at all incoming first-year residents as part of their required orientation. We conducted two sessions with approximately 55 interns and 5 volunteer faculty facilitators in each session. The study was approved for exempt status by the University of Nebraska Medical Center IRB and complied with the principles of the Declaration of Helsinki.

Workshop design

We developed the workshop curriculum based upon principles of published handoff pedagogy. The learning objectives were to 1) explain how handoffs impact patient safety, 2) list key elements for effective handoff communication, 3) perform a standardized strategy for written and verbal handoff communication, 4) perform “if-then” statements in the context of handoff communication, 5) demonstrate a “read-back” strategy in the context of handoff communication, and 6) demonstrate the role of patient prioritization when performing handoffs.

The training was comprised of brief didactic lectures, group discussion, and role-play activities. Throughout the training, we stressed the importance of providing unambiguous follow-up tasks and contingency planning for anticipated problems, prioritization of patients, and closed-loop communication. Resident dyads practiced written and verbal handoff skills via vignette-based role-play scenarios with direct faculty observation. Participants practiced written handoff skills by completing a template similar to what is used in the electronic health record and verbal handoff skills using a modified Situation Background Assessment Recommendations (SBAR) strategy, in which we added Questions and Read-back as required elements (SBAR-QR). The Appendix contains a more detailed description of course content and materials, including the moderator key.

Assessment

We conducted paired pre/post-intervention assessments immediately before and after the workshop to evaluate Kirkpatrick Level 2a (attitudes and confidence) and 2b (knowledge and skills) outcomes. Questions were based upon the expert opinions of the authors. To ensure content validity, a panel of local experts reviewed the survey, and it was pilot-tested for clarity with four senior medical students prior to distribution. We collected information on participants' baseline characteristics, including residency discipline, location of medical school (internal vs external graduate), previous handoff training, the number of handoffs previously observed by the participant, and the number of handoffs previously performed by the participant (none, 1–5, 6–10, 11–20, ≥20). Participants rated attitude-based questions using a 5-point Likert scale (1 = strongly disagree to 5 = strongly agree). To assess pre/post-intervention skills acquisition, we asked interns to provide an example of both a follow-up task and contingency plan that might occur during a high-quality handoff as a representative measure of skills acquisition. Two internal medicine physicians, who were not members of the research team, scored these independently on a predetermined 0–4 point scoring system with disagreements settled by consensus.

Analysis

We entered hardcopy survey data into a spreadsheet software program and calculated descriptive statistics. Incomplete surveys were excluded from the analysis. We analyzed paired pre/post-surveys using Wilcoxon signed-rank test for dependent samples. Comparisons of pre- and post-surveys were calculated in aggregate and based on the participant-level characteristics described above. For analysis purposes, we categorized residency disciplines as surgical or nonsurgical. We calculated Cronbach's α coefficient to assess the inter-rater reliability of the skills assessment scoring. We used IBM SPSS version 22 software for all analysis and considered a P-value ≤0.05 to be significant.

Results

Ninety-nine of 108 interns (91.7%) completed both pre-and post-surveys. Table 1 displays participants' baseline characteristic. Interns from 14 different residency programs participated in the workshop, with 53.5% coming from primary care disciplines. One-third (33/99) reported previous handoff training, mostly in the form of stand-alone workshops (11/33; 33%) or lectures (8/33; 24%). Seventy-five percent (74/99) of interns had observed >10 handoffs and 25% (25/99) had performed >10 handoffs.

Table 1.

Characteristics of 99 interns participating in handoff communication workshop.

	N	FREQUENCY (%)
Residency program
Anesthesiology	10	10.1
Emergency medicine	9	9.1
ENT surgery	2	2.0
Family medicine	17	17.2
Internal medicine	22	22.2
Internal medicine-pediatrics	2	2.0
Neurology	3	3.0
Obstetrics gynecology	4	4.0
OMF surgery	2	2.0
Orthopedic surgery	3	3.0
Pediatrics	12	12.1
Psychiatry	6	6.1
General surgery	5	5.1
Urology	2	2.0
Graduate of UNMC
No	64	64.6
Yes	35	35.4
Previous handoff training
No	66	66.7
Yes	33	33.3
Handoffs observed
None	1	1.0
1–5	13	13.1
6–10	11	11.1
11–20	16	16.2
>20	58	58.6
Handoffs performed
None	27	27.3
1–5	25	25.3
6–10	22	22.2
11–20	10	10.1
>20	15	15.2

Abbreviations: ENT, Ear, Nose, and Throat; OMF, Oral and Maxillofacial; UNMC, university of Nebraska Medical Center.

There was significant improvement in all 10 attitude-based questions (Table 2). With few exceptions, results remained significant regardless of prior training, number of handoffs observed, number of handoffs performed, medical school, or residency discipline. The primary exception was for question 10 (“handoff communication is important for patient safety”), which showed a significant change only for participants without prior training (z = -2.64; P = 0.049), 1–5 handoffs observed (z = -2.38; P = 0.016), or 1–5 handoffs performed (z = -2.43; P = 0.002). Workshop participants also demonstrated improved ability to construct follow-up task and contingency plans (pre-intervention 1.07, SD 1.03 vs post-intervention 2.16, SD 1.25; P ≤ 0.001; Cronbach's α = 0.88). These results remained significant despite prior training (prior training: z = -3.92; P ≤ 0.001; no prior training: z = -5.433; P ≤ 0.001).

Table 2.

Self-reported responses of 99 interns Before and After a 1-Hour Handoff Workshop.

	PRE-	POST-	Z	P
1. I know the critical elements of handoff communication	3.29	4.40	-8.144	≤.001
2. I can create effective written handoff communication	3.03	4.33	-9.056	≤.001
3. I know a standardized method for verbal handoff communication	2.77	4.46	-9.111	≤.001
4. I can effectively give verbal handoff communication	3.02	4.19	-8.282	≤.001
5. I can effectively receive verbal handoff communication	3.35	4.29	-7.330	≤.001
6. I know how to use “read-backs” in verbal handoff communication	3.06	4.45	-8.719	≤.001
7. I know how to make contingency plans for my patients	3.00	4.15	-8.070	≤.001
8. I can efficiently handoff patients	2.66	3.98	-9.166	≤.001
9. I am comfortable providing cross-cover care for patients	2.54	3.76	-8.776	≤.001
10. Handoff communication is important for patient safety	4.54	4.86	-2.457	.014

Discussion

We show that a brief, evidenced-based educational intervention is effective in improving interns' self-assessment of handoff competency and their ability to perform core handoff activities regardless of their prior handoff experiences. This study adds to the literature, as our findings support the need for handoff training for all interns immediately before beginning their residency programs. Such training serves two interrelated purposes. First, it provides a basic skill-set which can be enhanced and refined by individual residency programs depending on their discipline-specific needs. Second, it allows education about institution-specific expectations, policies, and practices, which can differ greatly between hospitals. This is important, as the ACGME's Clinical Learning Environment Review (CLER) Program requires sponsoring institutions demonstrate effective standardization of care transitions as part of its institutional accreditation system.

The fact that interns with and without prior training and real-world practice benefited equally from the handoff training is likely multifactorial. Although some trainees observed or performed handoffs as medical students, this does not guarantee that their behaviors were informed by principles of effective handoff practices. Additionally, skills decay following a period of latency between completing medical school and starting residency may have negated some of the benefit of prior experiences.

These findings should be interpreted cautiously. Our study was conducted at a single institution, so results may not be generalizable. We used a novel survey instrument, for which validity and reliability are uncertain, although inter-rater reliability for the skills assessment was good (Cronbach's α = 0.88). Additionally, assessment of handoff skills was limited to the residents' ability to effectively document a follow-up task and contingency plan, which may not be representative of other necessary skills, and scorers were not strictly blinded to whether the surveys were pre- or post-intervention. In future work, we hope to determine whether these initial improvements translate to clinical practice and whether the gains are retained in the following months.

Conclusion

Medical school graduates have varying backgrounds in handoff communication, and it is important for residency programs to ensure that incoming interns are competent to perform unsupervised handoffs. We found that a simple one-hour interactive workshop directed at incoming interns can improve participants' confidence and performance of basic handoff skills, regardless of previous training or experience.

Author Contributions

Conceived and designed the experiments: CS, MW, JH, GB. Analyzed the data: CS, MW, JH, GB. Wrote the first draft of the manuscript: CS. Contributed to the writing of the manuscript: CS, MW, JH, GB. Agree with manuscript results and conclusions: CS, MW, JH, GB. Jointly developed the structure and arguments for the paper: CS, MW, JH, GB. Made critical revisions and approved final version: CS, MW, JH, GB. All authors reviewed and approved of the final manuscript.

Supplementary Material

Appendix

Handoff workshop for incoming residents— course content and materials.