Reed G Williams1, Gary L Dunnington. 1. Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL 62794-9638, USA. rwilliams@siumed.edu
Abstract
BACKGROUND: This study investigated the concurrent and predictive validity of end-of-rotation (EOR) clinical performance ratings. STUDY DESIGN: Surgeon EOR ratings of residents were collected and compared with end-of-year (EOY) progress decisions and to EOR and EOY confidential judgments of resident ability to provide patient care without direct supervision. RESULTS: Eighty percent to 85% of EOR ratings were Excellent or Very Good. Five percent or fewer were Fair or Poor. Almost all residents receiving Excellent or Very Good EOR ratings also received positive EOR judgments about ability to provide patient care without direct supervision. Residents rated Fair or Poor received negative EOR judgments about ability to provide patient care without direct supervision. As the cumulative percent of Good, Fair, and Poor EOR ratings increased, the number of residents promoted without stipulations at the end of the year decreased and the percentage of faculty members who judged the residents capable of providing effective patient care without direct supervision at the end of the year declined. All residents receiving 40% or more EOR ratings below Very Good had stipulations associated with their promotion. CONCLUSIONS: Despite use of descriptive anchors on the scale, clinical performance ratings have no direct meaning. Their meaning needs to be established in the same manner as is done in setting normal values for diagnostic tests, ie, by establishing the relationship between EOR ratings and practice outcomes.
BACKGROUND: This study investigated the concurrent and predictive validity of end-of-rotation (EOR) clinical performance ratings. STUDY DESIGN: Surgeon EOR ratings of residents were collected and compared with end-of-year (EOY) progress decisions and to EOR and EOY confidential judgments of resident ability to provide patient care without direct supervision. RESULTS: Eighty percent to 85% of EOR ratings were Excellent or Very Good. Five percent or fewer were Fair or Poor. Almost all residents receiving Excellent or Very Good EOR ratings also received positive EOR judgments about ability to provide patient care without direct supervision. Residents rated Fair or Poor received negative EOR judgments about ability to provide patient care without direct supervision. As the cumulative percent of Good, Fair, and Poor EOR ratings increased, the number of residents promoted without stipulations at the end of the year decreased and the percentage of faculty members who judged the residents capable of providing effective patient care without direct supervision at the end of the year declined. All residents receiving 40% or more EOR ratings below Very Good had stipulations associated with their promotion. CONCLUSIONS: Despite use of descriptive anchors on the scale, clinical performance ratings have no direct meaning. Their meaning needs to be established in the same manner as is done in setting normal values for diagnostic tests, ie, by establishing the relationship between EOR ratings and practice outcomes.
Authors: M J B Govaerts; L W T Schuwirth; C P M Van der Vleuten; A M M Muijtjens Journal: Adv Health Sci Educ Theory Pract Date: 2010-09-30 Impact factor: 3.853