BACKGROUND: The ACGME requires the assessment of resident competency in 6 domains. Global evaluations covering all 6 competencies are routinely used. Evaluators may be overly influenced by resident affability and availability, thereby resulting in a halo effect. We hypothesized that the Interpersonal Skills and Communications (ICS) and Professionalism (PR) competencies would unduly influence other competency scores. METHODS: General surgery resident evaluations are performed by staff and peers on a rotational basis using competency-based questions. Each question is scored using a 5-point Likert scale. Mean individual composite scores for each competency were calculated and then correlated with other mean composite competency scores. Data from patient evaluations were similarly analyzed. A final correlation of competency scores to ABSITE scores, as an objective, standardized measure of a specific competency, Medical knowledge (MK) was also performed. RESULTS: Results were available for 37 residents (PGY 1-5). There was a significant association between ICS scores and higher scores in MK (r = 0.52, p = 0.004), PR (r = 0.826, p < 0.0001) and patient care (PC) (r = 0.619, p < 0.0001). No correlation, however, was found between patient evaluations of residents and their faculty/peer-based ICS scores. We found no association between ICS scores and improved patient evaluations. Lastly, we found no association between ICS or MK scores and ABSITE scores. CONCLUSIONS: It was difficult to ascertain whether residents with better ICS scores had higher PR, PC, and MK scores because of the halo effect, improper completion of evaluations, or whether those residents were truly performing better clinically. External measures of resident performance did not correlate with faculty/peer evaluations of ICS and PR. Residency programs should consider adopting a more standardized way to objectively evaluate residents.
BACKGROUND: The ACGME requires the assessment of resident competency in 6 domains. Global evaluations covering all 6 competencies are routinely used. Evaluators may be overly influenced by resident affability and availability, thereby resulting in a halo effect. We hypothesized that the Interpersonal Skills and Communications (ICS) and Professionalism (PR) competencies would unduly influence other competency scores. METHODS: General surgery resident evaluations are performed by staff and peers on a rotational basis using competency-based questions. Each question is scored using a 5-point Likert scale. Mean individual composite scores for each competency were calculated and then correlated with other mean composite competency scores. Data from patient evaluations were similarly analyzed. A final correlation of competency scores to ABSITE scores, as an objective, standardized measure of a specific competency, Medical knowledge (MK) was also performed. RESULTS: Results were available for 37 residents (PGY 1-5). There was a significant association between ICS scores and higher scores in MK (r = 0.52, p = 0.004), PR (r = 0.826, p < 0.0001) and patient care (PC) (r = 0.619, p < 0.0001). No correlation, however, was found between patient evaluations of residents and their faculty/peer-based ICS scores. We found no association between ICS scores and improved patient evaluations. Lastly, we found no association between ICS or MK scores and ABSITE scores. CONCLUSIONS: It was difficult to ascertain whether residents with better ICS scores had higher PR, PC, and MK scores because of the halo effect, improper completion of evaluations, or whether those residents were truly performing better clinically. External measures of resident performance did not correlate with faculty/peer evaluations of ICS and PR. Residency programs should consider adopting a more standardized way to objectively evaluate residents.
Authors: Laura R Hopson; Michele L Dorfsman; Jeremy Branzetti; Michael A Gisondi; Danielle Hart; Jaime Jordan; James A Cranford; Sarah R Williams; Linda Regan Journal: AEM Educ Train Date: 2019-05-20