INTRODUCTION: Technical skill has been formally assessed in the Fellow of the European Board of Vascular Surgery Examinations (FEBVS) since 2002. The aim of this study was to examine the relationship between expert assessment and trainee self-assessment. MATERIALS AND METHODS: Forty-two examination candidates performed a saphenofemoral junction (SFJ) ligation and an anterior tibial anastomosis on a synthetic simulation. Each candidate was rated by two examiners using a validated rating scale for their generic surgical skill for both procedures. Candidates then anonymously rated their own performance using the same scale. Parametric tests were used in the statistical analysis; a P-value < 0.05 was considered significant. RESULTS: The maximum mark in each assessment was 40; 24 was considered a competent score. The interobserver correlation for examiners marks were high (SFJ ligation, alpha = 0.68; distal anastomosis, alpha = 0.76). Examiners' marks were averaged. The mean examiner score for the SFJ ligation station was 27.8 (SD = 4.1) with 36 candidates (85.8%) attaining a competent score. The mean self-assessment score for this station was 30.7 (SD = 4.66). The mean examiners' marks for the distal anastomosis station was 29.2 (SD = 4.2); 39 candidates (92.8%) attained a competent score. The mean self-assessment score was 32.1 (SD = 4.0). There was no correlation between examiner and self-assessment scores in either station (Pearson's correlation coefficient: SFJ, r = 0.045, P = NS); distal anastomosis, r = 0.089, P = NS). Bland and Altman plots assessed the agreement between examiner and self-assessment. These showed candidates marked themselves higher than examiners with a mean difference of 2.9 marks in each station. CONCLUSIONS: Candidates' self-assessment and expert independent assessment correlate poorly. Trainees overestimate their ability according to independent assessment; regular technical feedback during training is, therefore, essential.
INTRODUCTION: Technical skill has been formally assessed in the Fellow of the European Board of Vascular Surgery Examinations (FEBVS) since 2002. The aim of this study was to examine the relationship between expert assessment and trainee self-assessment. MATERIALS AND METHODS: Forty-two examination candidates performed a saphenofemoral junction (SFJ) ligation and an anterior tibial anastomosis on a synthetic simulation. Each candidate was rated by two examiners using a validated rating scale for their generic surgical skill for both procedures. Candidates then anonymously rated their own performance using the same scale. Parametric tests were used in the statistical analysis; a P-value < 0.05 was considered significant. RESULTS: The maximum mark in each assessment was 40; 24 was considered a competent score. The interobserver correlation for examiners marks were high (SFJ ligation, alpha = 0.68; distal anastomosis, alpha = 0.76). Examiners' marks were averaged. The mean examiner score for the SFJ ligation station was 27.8 (SD = 4.1) with 36 candidates (85.8%) attaining a competent score. The mean self-assessment score for this station was 30.7 (SD = 4.66). The mean examiners' marks for the distal anastomosis station was 29.2 (SD = 4.2); 39 candidates (92.8%) attained a competent score. The mean self-assessment score was 32.1 (SD = 4.0). There was no correlation between examiner and self-assessment scores in either station (Pearson's correlation coefficient: SFJ, r = 0.045, P = NS); distal anastomosis, r = 0.089, P = NS). Bland and Altman plots assessed the agreement between examiner and self-assessment. These showed candidates marked themselves higher than examiners with a mean difference of 2.9 marks in each station. CONCLUSIONS: Candidates' self-assessment and expert independent assessment correlate poorly. Trainees overestimate their ability according to independent assessment; regular technical feedback during training is, therefore, essential.
Authors: Mylène Ward; Helen MacRae; Christopher Schlachta; Joseph Mamazza; Eric Poulin; Richard Reznick; Glenn Regehr Journal: Am J Surg Date: 2003-06 Impact factor: 2.565
Authors: Paula Clifford; Natasha Melfi; John Bogdanske; Elizabeth J Johnson; James Kehler; Szczepan W Baran Journal: J Am Assoc Lab Anim Sci Date: 2013-11 Impact factor: 1.232
Authors: Mitchell B Alameddine; Jake Claflin; Christopher P Scally; David M Noble; Bradley N Reames; Michael J Englesbe; Sandra L Wong Journal: J Surg Educ Date: 2015-09-26 Impact factor: 2.891