Does the subjective evaluation of medical student surgical knowledge correlate with written and oral exam performance?

BACKGROUND: Medical student performance evaluations have historically contained a significant subjective component. Multiple tools are used to assess fund of knowledge including subjective evaluation by faculty and residents as well as objective evaluations through standardized written and oral exams. We hypothesized that subjective evaluation of medical student knowledge would correlate with objective evaluation through written and oral exams.
METHODS: Records of consecutive medical students assigned to the surgery clerkship from January 1999 and March 2001 were reviewed. The core surgical rotation consisted of two 4-week blocks on a private, county, or VA hospital service. Surgical knowledge was assessed subjectively by both faculty (FES) and senior residents (RES) using a 10-point scale with verbal anchors. Objective measures of student surgical knowledge included the National Board shelf exam (WE) and a semistructured oral exam (OE). Data are reported as mean +/- SEM. Spearman rank correlation coefficient (r) was used to assess relationships between groups (r > or = 0.5 --> positive correlation).
RESULTS: A total of 354 students were evaluated. The mean FES was 7.8 +/- 0.05 (median = 7.75, range 4.75 to 9.75). The mean RES was 7.7 +/- 0.06 (median = 8.0, range 3.5 to 10.0). There was poor correlation between the subjective perception and objective measures of surgical knowledge (Table 1). Comparison of the FES and RES also showed poor correlation (r = 0.38).
CONCLUSIONS: Subjective evaluation of surgical knowledge by faculty and residents correlates poorly with performance measured objectively. These results question whether subjective evaluation of surgical knowledge should be included as part of the evaluation process.