BACKGROUND: The Kalamazoo Essential Elements Communication Checklist-Adapted (KEECC-A) is a well-regarded instrument for evaluating communication and interpersonal skills. To date, little research has been conducted that assesses the accuracy of resident self-ratings of their communication skills. OBJECTIVE: To assess whether residents can accurately self-rate communication skills, using the KEECC-A, during an objective structured clinical examination (OSCE). METHODS: A group of 104 residents from 8 specialties completed a multistation OSCE as part of an institutional communication skills curriculum conducted at a single institution. Standardized patients (SPs) and observers were trained in rating communication skills using the KEECC-A. Standardized patient ratings and resident self-ratings were completed immediately following each OSCE encounter, and trained observers rated archived videotapes of the encounters. RESULTS: Resident self-ratings and SP ratings using the KEECC-A were significantly correlated (r104 = 0.238, P = .02), as were resident self-ratings and observer ratings (r104 = 0.284, P = .004). The correlation between the SP ratings and observer (r104 = 0.378, P = .001) ratings were larger in magnitude, but not significantly different (P > .05) from resident/SP or resident/observer correlations. CONCLUSIONS: The results suggest that residents, with a modicum of training using the KEECC-A, can accurately rate their own communication and interpersonal skills during an OSCE. Using trained observers to rate resident communication skills provides a unique opportunity for evaluating SP and resident self-ratings. Our findings also lend further support for the reliability and validity of the KEECC-A.
BACKGROUND: The Kalamazoo Essential Elements Communication Checklist-Adapted (KEECC-A) is a well-regarded instrument for evaluating communication and interpersonal skills. To date, little research has been conducted that assesses the accuracy of resident self-ratings of their communication skills. OBJECTIVE: To assess whether residents can accurately self-rate communication skills, using the KEECC-A, during an objective structured clinical examination (OSCE). METHODS: A group of 104 residents from 8 specialties completed a multistation OSCE as part of an institutional communication skills curriculum conducted at a single institution. Standardized patients (SPs) and observers were trained in rating communication skills using the KEECC-A. Standardized patient ratings and resident self-ratings were completed immediately following each OSCE encounter, and trained observers rated archived videotapes of the encounters. RESULTS: Resident self-ratings and SP ratings using the KEECC-A were significantly correlated (r104 = 0.238, P = .02), as were resident self-ratings and observer ratings (r104 = 0.284, P = .004). The correlation between the SP ratings and observer (r104 = 0.378, P = .001) ratings were larger in magnitude, but not significantly different (P > .05) from resident/SP or resident/observer correlations. CONCLUSIONS: The results suggest that residents, with a modicum of training using the KEECC-A, can accurately rate their own communication and interpersonal skills during an OSCE. Using trained observers to rate resident communication skills provides a unique opportunity for evaluating SP and resident self-ratings. Our findings also lend further support for the reliability and validity of the KEECC-A.