OBJECTIVE: To evaluate the effectiveness of formal assessments of prevocational medical trainees (ie, interns, first-year residents and international medical graduates undergoing supervised training). DESIGN AND SETTING: Retrospective review of structured assessment forms that are completed by all prevocational trainees and their supervisors in New South Wales public hospitals. We reviewed the first 3390 assessment forms, representing assessments of 1072 trainees at 43 training sites (83% of all prevocational trainees in NSW) since January 2009. MAIN OUTCOME MEASURES: (i) Trainee ratings on 19 assessment items by self-assessment and by supervisor assessment; and (ii) quantity and quality of written comments provided in assessments. RESULTS: At the end of term, 43% of trainees assessed themselves as performing "at expected level" on all 19 rating items. Nearly 99% of trainees were assessed by their supervisors as performing at or above the expected level on all assessment items. Written comments by supervisors were generally short and encouraging, but lacked specific feedback that trainees might use to guide improvements in performance. CONCLUSIONS: As currently used by trainees and supervisors, the assessment forms may underreport trainee underperformance, do not discriminate strongly between different levels of performance of trainees or the training system, and do not provide trainees with enough specific feedback to guide their professional development.
OBJECTIVE: To evaluate the effectiveness of formal assessments of prevocational medical trainees (ie, interns, first-year residents and international medical graduates undergoing supervised training). DESIGN AND SETTING: Retrospective review of structured assessment forms that are completed by all prevocational trainees and their supervisors in New South Wales public hospitals. We reviewed the first 3390 assessment forms, representing assessments of 1072 trainees at 43 training sites (83% of all prevocational trainees in NSW) since January 2009. MAIN OUTCOME MEASURES: (i) Trainee ratings on 19 assessment items by self-assessment and by supervisor assessment; and (ii) quantity and quality of written comments provided in assessments. RESULTS: At the end of term, 43% of trainees assessed themselves as performing "at expected level" on all 19 rating items. Nearly 99% of trainees were assessed by their supervisors as performing at or above the expected level on all assessment items. Written comments by supervisors were generally short and encouraging, but lacked specific feedback that trainees might use to guide improvements in performance. CONCLUSIONS: As currently used by trainees and supervisors, the assessment forms may underreport trainee underperformance, do not discriminate strongly between different levels of performance of trainees or the training system, and do not provide trainees with enough specific feedback to guide their professional development.
Authors: Helen A Scicluna; Michael C Grimm; Philip D Jones; Louis S Pilotto; H Patrick McNeil Journal: BMC Med Educ Date: 2014-02-03 Impact factor: 2.463