BACKGROUND: Increasing frequency of shift-to-shift handoffs coupled with regulatory requirements to evaluate handoff quality make a handoff evaluation tool necessary. OBJECTIVE: To develop a handoff evaluation tool. DESIGN: Tool development. SETTING: Two academic medical centers. SUBJECTS: Nurse practitioners, medicine housestaff, and hospitalist attendings. INTERVENTION: Concurrent peer and external evaluations of shift-to-shift handoffs. MEASUREMENTS: The Handoff CEX (clinical evaluation exercise) consists of 6 subdomains and 1 overall assessment, each scored from 1 to 9, where 1 to 3 is unsatisfactory and 7 to 9 is superior. We assessed range of scores, performance among subgroups, internal consistency, and agreement among types of raters. RESULTS: We conducted 675 evaluations of 97 unique individuals during 149 handoff sessions. Scores ranged from unsatisfactory to superior in each domain. The highest rated domain for handoff providers was professionalism (median: 8; interquartile range [IQR]: 7-9); the lowest was content (median: 7; IQR: 6-8). Scores at the 2 institutions were similar, and scores did not differ significantly by training level. Spearman correlation coefficients among the CEX subdomains for provider scores ranged from 0.71 to 0.86, except for setting (0.39-0.40). Third-party external evaluators consistently gave lower marks for the same handoff than peer evaluators did. Weighted kappa scores for provider evaluations comparing external evaluators to peers ranged from 0.28 (95% confidence interval [CI]: 0.01, 0.56) for setting to 0.59 (95% CI: 0.38, 0.80) for organization. CONCLUSIONS: This handoff evaluation tool was easily used by trainees and attendings, had high internal consistency, and performed similarly across institutions. Because peers consistently provided higher scores than external evaluators, this tool may be most appropriate for external evaluation.
BACKGROUND: Increasing frequency of shift-to-shift handoffs coupled with regulatory requirements to evaluate handoff quality make a handoff evaluation tool necessary. OBJECTIVE: To develop a handoff evaluation tool. DESIGN: Tool development. SETTING: Two academic medical centers. SUBJECTS: Nurse practitioners, medicine housestaff, and hospitalist attendings. INTERVENTION: Concurrent peer and external evaluations of shift-to-shift handoffs. MEASUREMENTS: The Handoff CEX (clinical evaluation exercise) consists of 6 subdomains and 1 overall assessment, each scored from 1 to 9, where 1 to 3 is unsatisfactory and 7 to 9 is superior. We assessed range of scores, performance among subgroups, internal consistency, and agreement among types of raters. RESULTS: We conducted 675 evaluations of 97 unique individuals during 149 handoff sessions. Scores ranged from unsatisfactory to superior in each domain. The highest rated domain for handoff providers was professionalism (median: 8; interquartile range [IQR]: 7-9); the lowest was content (median: 7; IQR: 6-8). Scores at the 2 institutions were similar, and scores did not differ significantly by training level. Spearman correlation coefficients among the CEX subdomains for provider scores ranged from 0.71 to 0.86, except for setting (0.39-0.40). Third-party external evaluators consistently gave lower marks for the same handoff than peer evaluators did. Weighted kappa scores for provider evaluations comparing external evaluators to peers ranged from 0.28 (95% confidence interval [CI]: 0.01, 0.56) for setting to 0.59 (95% CI: 0.38, 0.80) for organization. CONCLUSIONS: This handoff evaluation tool was easily used by trainees and attendings, had high internal consistency, and performed similarly across institutions. Because peers consistently provided higher scores than external evaluators, this tool may be most appropriate for external evaluation.
Authors: Jeanne M Farnan; J A M Paro; R M Rodriguez; S T Reddy; L I Horwitz; J K Johnson; V M Arora Journal: J Gen Intern Med Date: 2010-02 Impact factor: 5.128
Authors: Gregory M Bump; Franziska Jovin; Lindsay Destefano; Amanda Kirlin; Andrew Moul; Kelly Murray; Deborah Simak; D Michael Elnicki Journal: Teach Learn Med Date: 2011-04 Impact factor: 2.414
Authors: Monica K Lieng; Heather M Siefkes; Jennifer L Rosenthal; Hadley S Sauers-Ford; Jamie L Mouzoon; Ilana S Sigal; Parul Dayal; Shelby T Chen; Cheryl L McBeth; Sandie Dial; Genevieve Dizon; Haley E Dannewitz; Kiersten Kozycz; Torryn L Jennings-Hill; Jennifer M Martinson; Julia K Huerta; Emily A Pons; Nicole Vance; Breanna N Warnock; James P Marcin Journal: Pediatr Crit Care Med Date: 2019-09 Impact factor: 3.624
Authors: Katherine E Bates; Geoffrey L Bird; Judy A Shea; Michael Apkon; Robert E Shaddy; Joshua P Metlay Journal: J Hosp Med Date: 2014-01-31 Impact factor: 2.960
Authors: Sean Gaffney; Jeanne M Farnan; Kristen Hirsch; Michael McGinty; Vineet M Arora Journal: J Gen Intern Med Date: 2016-01-29 Impact factor: 5.128
Authors: Vineet M Arora; Saba Berhie; Leora I Horwitz; Mark Saathoff; Paul Staisiunas; Jeanne M Farnan Journal: J Hosp Med Date: 2014-03-24 Impact factor: 2.960
Authors: Juan A Reyes; Larrie Greenberg; Richard Amdur; James Gehring; Linda G Lesky Journal: Adv Health Sci Educ Theory Pract Date: 2015-07-15 Impact factor: 3.853