Jason J Lewis1, Lakshman Balaji2, Anne V Grossestreuer2, Edward Ullman1, Carlo Rosen1, Nicole M Dubosh1. 1. Department of Emergency Medicine Beth Israel Deaconess Medical Center/Harvard Affiliated Emergency Medicine Residency Boston Massachusetts USA. 2. Department of Emergency Medicine Beth Israel Deaconess Medical Center Boston Massachusetts USA.
Abstract
BACKGROUND: Communication and interpersonal skills are one of the Accreditation Council for Graduate Medical Education's six core competencies. Validated methods for assessing these among trainees are lacking. Educators have developed various communication assessment tools from both the supervising attending and the patient perspectives. How these different assessment methods and tools compare with each other remains unknown. The goal of this study was to determine the degree of agreement between attending and patient assessment of resident communication skills. METHODS: This was a retrospective study of emergency medicine (EM) residents at an academic medical center. From July 2017 to June 2018, residents were assessed on communication skills during their emergency department shifts by both their supervising attending physicians and their patients. The attendings rated residents' communication skills with patients, colleagues, and nursing/ancillary staff using a 1 to 5 Likert scale. Patients completed the modified Communication Assessment Tool (CAT), a 14-item questionnaire based on a 1 to 5 Likert scale. Mean attending ratings and patient CAT scores were calculated for each resident. Means were divided into tertiles due to nonparametric distribution of scores. Agreement between attending and patient ratings of residents were measured using Cohen's kappa for each attending evaluation question. Scores were weighted to assign adjacent tertiles partial agreement. RESULTS: During the study period, 1,097 attending evaluations and 952 patient evaluations were completed for 26 residents. Attending scores and CAT scores of the residents showed slight to fair agreement in the following three domains: patient communication (κ = 0.21), communication with colleagues (κ = 0.21), and communication with nursing/ancillary staff (κ = 0.26). CONCLUSIONS: Attending and patient ratings of EM residents' communication skills show slight to fair agreement. The use of different types of raters may be beneficial in fully assessing trainees' communication skills.
BACKGROUND: Communication and interpersonal skills are one of the Accreditation Council for Graduate Medical Education's six core competencies. Validated methods for assessing these among trainees are lacking. Educators have developed various communication assessment tools from both the supervising attending and the patient perspectives. How these different assessment methods and tools compare with each other remains unknown. The goal of this study was to determine the degree of agreement between attending and patient assessment of resident communication skills. METHODS: This was a retrospective study of emergency medicine (EM) residents at an academic medical center. From July 2017 to June 2018, residents were assessed on communication skills during their emergency department shifts by both their supervising attending physicians and their patients. The attendings rated residents' communication skills with patients, colleagues, and nursing/ancillary staff using a 1 to 5 Likert scale. Patients completed the modified Communication Assessment Tool (CAT), a 14-item questionnaire based on a 1 to 5 Likert scale. Mean attending ratings and patient CAT scores were calculated for each resident. Means were divided into tertiles due to nonparametric distribution of scores. Agreement between attending and patient ratings of residents were measured using Cohen's kappa for each attending evaluation question. Scores were weighted to assign adjacent tertiles partial agreement. RESULTS: During the study period, 1,097 attending evaluations and 952 patient evaluations were completed for 26 residents. Attending scores and CAT scores of the residents showed slight to fair agreement in the following three domains: patient communication (κ = 0.21), communication with colleagues (κ = 0.21), and communication with nursing/ancillary staff (κ = 0.26). CONCLUSIONS: Attending and patient ratings of EM residents' communication skills show slight to fair agreement. The use of different types of raters may be beneficial in fully assessing trainees' communication skills.
Authors: Rebecca Anhang Price; Marc N Elliott; Alan M Zaslavsky; Ron D Hays; William G Lehrman; Lise Rybowski; Susan Edgman-Levitan; Paul D Cleary Journal: Med Care Res Rev Date: 2014-07-15 Impact factor: 3.929
Authors: Mohammadreza Hojat; Daniel Z Louis; Fred W Markham; Richard Wender; Carol Rabinowitz; Joseph S Gonnella Journal: Acad Med Date: 2011-03 Impact factor: 6.893
Authors: Scott R Millis; Sudesh Sheela Jain; Mary Eyles; David Tulsky; Scott F Nadler; Patrick M Foye; Elie Elovic; Joel A DeLisa Journal: Am J Phys Med Rehabil Date: 2002-12 Impact factor: 2.159
Authors: Amber W Trickey; Anna B Newcomb; Melissa Porrey; Jeffrey Wright; Jordan Bayless; Franco Piscitani; Paula Graling; Jonathan Dort Journal: J Surg Educ Date: 2016-05-20 Impact factor: 2.891