J D Beard1, S Choksy, S Khan. 1. Sheffield Vascular Institute, Northern General Hospital, Sheffield S5 7AU, UK. Jonathan.D.Beard@sth.nhs.uk
Abstract
BACKGROUND: Objective Structured Assessment of Technical Skills (OSATS) has been used to assess operative competence. The aim was to evaluate its role in carotid endarterectomy (CEA). METHODS: Over 2 years, 17 trainees and 11 consultants were assessed by a single observer using a task-specific checklist and global ratings. Operating time and previous experience were also recorded. RESULTS: Trainees had previously performed fewer CEAs than consultants (median (interquartile range, i.q.r.) 15 (7-24) versus 94 (61-133) respectively; P < 0.001) and they took longer to perform operations (median (i.q.r.) 128 (119-143) versus 90 (82.5-98.5) min; P < 0.001). Trainees achieved lower task-specific (median (i.q.r.) 23 (17-26) versus 25 (24.5-27.5); P = 0.031) and global (33 (30-35) versus 40 (36.5-43); P = 0.001) scores than consultants. There was a strong positive correlation between task-specific scores and experience for trainees (r = 0.83, P < 0.001), but not for consultants. Global ratings correlated with experience for trainees (r = 0.69, P = 0.002) and more so for consultants (r = 0.82, P = 0.002). There was a positive correlation between task-specific and global scores for trainees (r = 0.67, P = 0.003), but not for consultants. CONCLUSION: Task-specific checklists were more discriminatory between trainees and may be useful for formative assessment (training). Global ratings were more discriminatory for consultants and may be useful for summative assessment (examination or revalidation). (c) 2007 British Journal of Surgery Society Ltd.
BACKGROUND: Objective Structured Assessment of Technical Skills (OSATS) has been used to assess operative competence. The aim was to evaluate its role in carotid endarterectomy (CEA). METHODS: Over 2 years, 17 trainees and 11 consultants were assessed by a single observer using a task-specific checklist and global ratings. Operating time and previous experience were also recorded. RESULTS: Trainees had previously performed fewer CEAs than consultants (median (interquartile range, i.q.r.) 15 (7-24) versus 94 (61-133) respectively; P < 0.001) and they took longer to perform operations (median (i.q.r.) 128 (119-143) versus 90 (82.5-98.5) min; P < 0.001). Trainees achieved lower task-specific (median (i.q.r.) 23 (17-26) versus 25 (24.5-27.5); P = 0.031) and global (33 (30-35) versus 40 (36.5-43); P = 0.001) scores than consultants. There was a strong positive correlation between task-specific scores and experience for trainees (r = 0.83, P < 0.001), but not for consultants. Global ratings correlated with experience for trainees (r = 0.69, P = 0.002) and more so for consultants (r = 0.82, P = 0.002). There was a positive correlation between task-specific and global scores for trainees (r = 0.67, P = 0.003), but not for consultants. CONCLUSION: Task-specific checklists were more discriminatory between trainees and may be useful for formative assessment (training). Global ratings were more discriminatory for consultants and may be useful for summative assessment (examination or revalidation). (c) 2007 British Journal of Surgery Society Ltd.
Authors: Sarah E Peyre; Christian G Peyre; Jeffrey A Hagen; Maura E Sullivan; John C Lipham; Steven R Demeester; Jeffrey H Peters; Tom R Demeester Journal: Surg Endosc Date: 2008-12-05 Impact factor: 4.584