Andrea Wendling1, Philip Baty. 1. Department of Family Medicine, Michigan State University, Charlevoix, MI 49720, USA. aharmeling@msn.com
Abstract
BACKGROUND: Assessing the baseline competency of incoming interns is a challenge to faculty of all residency programs. This article describes and evaluates the Intern Clinical Judgment Evaluation, a tool developed to test the clinical judgment and communication skills of incoming interns. METHODS: During orientation week, each intern is evaluated using structured clinical scenarios modeled after "on call" situations. Interns are given feedback on information collection, medical decision making, judgment, understanding of the medical system, and written and verbal communication skills. Data from 4 years is presented, comparing predictions of performance from faculty and from the Intern Clinical Judgment Evaluation to actual year-end intern performance data. RESULTS: There was excellent agreement between the Intern Clinical Judgment Evaluation and the summary of interns' performance (kappa value=0.8). Faculty predictions were a less accurate predictor of intern performance (kappa value=0.6). The Intern Clinical Judgment Evaluation had a sensitivity of 93% (13/14), specificity of 88% (15/17), positive likelihood ratio (LR+) of 7.9, and negative likelihood ratio (LR-) of 0.08. Comparatively, faculty predictions of resident performance had a sensitivity of 57% (8/14), specificity of 94% (17/18), LR+ of 10.3, and LR- of 0.40. Surveyed interns felt the evaluation process was a positive experience. CONCLUSIONS: The Intern Clinical Judgment Evaluation was a valid tool to predict which interns would require more faculty supervision during intern year. Based on solicited feedback from tested interns, it also was a positive testing experience.
BACKGROUND: Assessing the baseline competency of incoming interns is a challenge to faculty of all residency programs. This article describes and evaluates the Intern Clinical Judgment Evaluation, a tool developed to test the clinical judgment and communication skills of incoming interns. METHODS: During orientation week, each intern is evaluated using structured clinical scenarios modeled after "on call" situations. Interns are given feedback on information collection, medical decision making, judgment, understanding of the medical system, and written and verbal communication skills. Data from 4 years is presented, comparing predictions of performance from faculty and from the Intern Clinical Judgment Evaluation to actual year-end intern performance data. RESULTS: There was excellent agreement between the Intern Clinical Judgment Evaluation and the summary of interns' performance (kappa value=0.8). Faculty predictions were a less accurate predictor of intern performance (kappa value=0.6). The Intern Clinical Judgment Evaluation had a sensitivity of 93% (13/14), specificity of 88% (15/17), positive likelihood ratio (LR+) of 7.9, and negative likelihood ratio (LR-) of 0.08. Comparatively, faculty predictions of resident performance had a sensitivity of 57% (8/14), specificity of 94% (17/18), LR+ of 10.3, and LR- of 0.40. Surveyed interns felt the evaluation process was a positive experience. CONCLUSIONS: The Intern Clinical Judgment Evaluation was a valid tool to predict which interns would require more faculty supervision during intern year. Based on solicited feedback from tested interns, it also was a positive testing experience.