The 360-degree evaluation: increased work with little return?

OBJECTIVE: To test a 360-degree resident evaluation tool on our trauma/critical care services to determine if multiple raters yielded equivalent information compared with traditional faculty evaluations.
DESIGN: Prospective evaluation. PARTICIPANTS: Residents, nurses, faculty, and staff at an academic medical center.
METHODS: The evaluation tool was developed based on extensive qualitative analysis of 13 major medical specialties' Residency Review Committee (RRC) criteria relative to the ACGME competencies and then revised with content specific to surgery. The evaluation contained 19 items divided into ACGME competency areas. Each item was scored on a 1 to 9 Likert scale: 1 = not meeting expectations and 9 = exceeding expectations. Residents on the trauma and surgical intensive care unit rotations evaluated themselves, and they were also evaluated by chief residents, surgical intensive care unit fellows, faculty, surgical intensive care unit nurses, trauma nurse clinicians, and nurse practitioners. Multiple analyses of variance were used to compare ratings by rater groups.
RESULTS: Ten residents were evaluated on the trauma service from April to August 2003. Between 74 and 106 evaluations were obtained per resident per competency area. Average scores across the competencies were remarkably similar, ranging from 6.18 for practice-based learning and systems-based practice to 6.54 for professionalism. Although there was variability within rater groups, ratings were not statistically different between groups for any ACGME competency.
CONCLUSIONS: The 360-degree evaluation provide limited new information compared with traditional faculty ratings. Follow-up studies are required to confirm this finding with larger samples of residents and surgical specialties.