INTRODUCTION: Selection of candidates for surgical fellowships has traditionally been based on subjective evaluations by the program directors and references from previous positions. The introduction of well-validated objective methods of assessment has allowed us to evaluate candidates' technical skills and base the selection process on objective, reliable, and transparent criteria. The aim of the study was to assess the applicability of such methods in current practice. MATERIALS AND METHODS: Prospective study. Eight surgeons, applying for a fellowship position in minimally invasive surgery (MIS), performed a previously validated assessment curriculum using a Virtual-Reality Laparoscopic Trainer (LapSim 3.0, Surgical Science, Gothenburgh, Sweden). Technical performance was evaluated using criteria registered by the simulator, i.e., time, error score, and efficiency of movements score. Candidates performed all the tasks in easy end medium level until reaching predefined criteria. If proficiency criteria were not achieved on easy or medium level after nine repetitions the test was considered as failed. Additionally, all applicants underwent an interview by two independent attending surgeons. Each applicant received a grade on a ten-point scale. RESULTS: Five out of the eight candidates failed the technical skills assessment test. One candidate failed to achieve proficiency criteria on easy level, one on medium, and three on difficult level. Evaluation scores, based on the interview of the candidates showed a good interrater reliability (Cronbach's alpha = 0.8). There was no significant correlation between the interviewers rating, and the applicants technical skills demonstrated during the test on the VR trainer (Spearman's rho = 0.182, p = 0.696). CONCLUSIONS: Evaluations by senior surgeons are reproducible and reliable. The introduction of technical skills assessment has the potential to improve the current method of candidate selection, making it more valid, objective, and transparent.
INTRODUCTION: Selection of candidates for surgical fellowships has traditionally been based on subjective evaluations by the program directors and references from previous positions. The introduction of well-validated objective methods of assessment has allowed us to evaluate candidates' technical skills and base the selection process on objective, reliable, and transparent criteria. The aim of the study was to assess the applicability of such methods in current practice. MATERIALS AND METHODS: Prospective study. Eight surgeons, applying for a fellowship position in minimally invasive surgery (MIS), performed a previously validated assessment curriculum using a Virtual-Reality Laparoscopic Trainer (LapSim 3.0, Surgical Science, Gothenburgh, Sweden). Technical performance was evaluated using criteria registered by the simulator, i.e., time, error score, and efficiency of movements score. Candidates performed all the tasks in easy end medium level until reaching predefined criteria. If proficiency criteria were not achieved on easy or medium level after nine repetitions the test was considered as failed. Additionally, all applicants underwent an interview by two independent attending surgeons. Each applicant received a grade on a ten-point scale. RESULTS: Five out of the eight candidates failed the technical skills assessment test. One candidate failed to achieve proficiency criteria on easy level, one on medium, and three on difficult level. Evaluation scores, based on the interview of the candidates showed a good interrater reliability (Cronbach's alpha = 0.8). There was no significant correlation between the interviewers rating, and the applicants technical skills demonstrated during the test on the VR trainer (Spearman's rho = 0.182, p = 0.696). CONCLUSIONS: Evaluations by senior surgeons are reproducible and reliable. The introduction of technical skills assessment has the potential to improve the current method of candidate selection, making it more valid, objective, and transparent.
Authors: F J Carter; M P Schijven; R Aggarwal; T Grantcharov; N K Francis; G B Hanna; J J Jakimowicz Journal: Surg Endosc Date: 2005-10-26 Impact factor: 4.584
Authors: Barbara Goff; Lynn Mandel; Gretchen Lentz; Amy Vanblaricom; Anne-Marie Amies Oelschlager; David Lee; Andrew Galakatos; Matthew Davies; Peter Nielsen Journal: Am J Obstet Gynecol Date: 2005-04 Impact factor: 8.661
Authors: Rajesh Aggarwal; Teodor P Grantcharov; Jens R Eriksen; Dorthe Blirup; Viggo B Kristiansen; Peter Funch-Jensen; Ara Darzi Journal: Ann Surg Date: 2006-08 Impact factor: 12.969