Knowledge base evaluation of medicine residents on the gastroenterology service: Implications for competency assessments by faculty.

BACKGROUND AND AIMS: Clinician educators are asked to provide both formative and summative evaluations on the medical knowledge of residents. This study evaluated the accuracy of these evaluations and the perception of residents regarding the ability of faculty to assess medical knowledge.
METHODS: Gastroenterology knowledge ratings provided by 15 faculty gastroenterologists on 49 internal medicine residents during a required gastroenterology rotation were correlated with performance on the gastroenterology subsection of the In-Training Examination for Internal Medicine. Residents also were surveyed regarding their perception of the ability of faculty to judge their knowledge of medical gastroenterology.
RESULTS: The mean correlation (Kendall's tau b) of faculty ratings with performance on the ITE was 0.30 (P < 0.01). The range of correlation values for individual faculty (-0.39 to 0.80) indicated that some faculty were able to assess the medical knowledge of residents better than others. Residents, as well as the faculty themselves, perceived that faculty were able to rate their medical knowledge relatively well.
CONCLUSIONS: The ability of faculty gastroenterologists to judge the knowledge of gastroenterology in their resident trainees was quite limited. Residents, as well as faculty, inaccurately perceive the ability of gastroenterologists to render professional judgments on their knowledge base as good. An end-of-rotation written examination would appear to be required to provide an accurate assessment of the medical knowledge of residents.