BACKGROUND: The major goal of certification is to assure the public that the candidate is competent in all facets required of the position. The patient assessment and management examination (PAME) was developed to enable a more comprehensive assessment of competence in the practice of surgery. METHODS: A six-station, 3-hour, standardized-patient-based evaluation was developed. Each station was scored using a set of five-point global rating scales. PAME results were compared to the last two in training evaluation reports (ITER), the clinical knowledge component of the ITER (ITER-CK), an in-house oral examination (OE), and the Canadian Association of General Surgeons' multiple-choice examination (CAGS). RESULTS: Eighteen senior general surgery residents were evaluated. Overall reliability was 0.70 (Cronbach's alpha). Fifth-year residents scored significantly better than fourth-year residents (t = 3.062; p = 0.0074), with 1 year of training accounting for 37% of the variance in scores. Correlations between the PAME and each of the other measures were ITER, 0.24; ITER-CK, 0.38; OE, -0.13; and CAGS, 0.061, with the PAME demonstrating better reliability and stronger evidence of validity than any other. CONCLUSIONS: The PAME had better psychometric properties than other measures and assessed areas often not evaluated. This type of evaluation may be useful for feedback, remediation, or certification decisions.
BACKGROUND: The major goal of certification is to assure the public that the candidate is competent in all facets required of the position. The patient assessment and management examination (PAME) was developed to enable a more comprehensive assessment of competence in the practice of surgery. METHODS: A six-station, 3-hour, standardized-patient-based evaluation was developed. Each station was scored using a set of five-point global rating scales. PAME results were compared to the last two in training evaluation reports (ITER), the clinical knowledge component of the ITER (ITER-CK), an in-house oral examination (OE), and the Canadian Association of General Surgeons' multiple-choice examination (CAGS). RESULTS: Eighteen senior general surgery residents were evaluated. Overall reliability was 0.70 (Cronbach's alpha). Fifth-year residents scored significantly better than fourth-year residents (t = 3.062; p = 0.0074), with 1 year of training accounting for 37% of the variance in scores. Correlations between the PAME and each of the other measures were ITER, 0.24; ITER-CK, 0.38; OE, -0.13; and CAGS, 0.061, with the PAME demonstrating better reliability and stronger evidence of validity than any other. CONCLUSIONS: The PAME had better psychometric properties than other measures and assessed areas often not evaluated. This type of evaluation may be useful for feedback, remediation, or certification decisions.