M P Schijven1, J Jakimowicz, C Schot. 1. Department of Surgery, Catharina Hospital Eindhoven, Michelangelolaan 2, P.O. Box 1350, 5602 ZA Eindhoven, The Netherlands. mschijv@zonnet.nl
Abstract
BACKGROUND: This study was undertaken to establish the value of the Advanced Dundee Psychomotor Tester (ADEPT) as an objective real-time scoring system, correcting for subjective assessor opinion on endoscopic task performance. The main research questions were as follows: Are surgeons good estimators of their own performance on ADEPT? Do surgeons perceive ADEPT to be a valid instrument for measuring laparoscopic skills? Does performance on ADEPT reflect innate psychomotor ability? METHODS: Each of 45 surgeons completed two runs on ADEPT. The runs comprised five standardized tasks. A posttest visual analog scaled (VAS) questionnaire measuring attitude toward skills testing in general, validation, and performance on ADEPT was used. Subjective responses were compared with objective scores generated through performance on ADEPT. RESULTS: Surgeons emphasize the importance of using a variety of training methods for surgical residents during their residency, including laparoscopic virtual reality simulators. Monitoring of residents' endoscopic progress seemed to be a key issue. Surgeons themselves underestimate their individual performance on ADEPT (mean subjective score of 6.1 vs mean objective score of 6.6). Self-reported performance on ADEPT is unreliable because confidence intervals between the VAS score and the ADEPT score overlap. Surgeons disagree on the validity of ADEPT. The mean score for validity was 5.8, ranging from 0 to 10 with almost equal distribution over the scale. Innate ability is established as surgeons' scores express high concordance between test run and true run, with 72.7% of the participants' true run score within one distance from the test run. CONCLUSIONS: Surgeons cannot correctly predict their standardized individual test result on ADEPT. Performance on ADEPT reflects innate psychomotor ability along with improvement over runs. Surgeons are ambivalent in assessing the validity of ADEPT, irrespective of personal performance.
BACKGROUND: This study was undertaken to establish the value of the Advanced Dundee Psychomotor Tester (ADEPT) as an objective real-time scoring system, correcting for subjective assessor opinion on endoscopic task performance. The main research questions were as follows: Are surgeons good estimators of their own performance on ADEPT? Do surgeons perceive ADEPT to be a valid instrument for measuring laparoscopic skills? Does performance on ADEPT reflect innate psychomotor ability? METHODS: Each of 45 surgeons completed two runs on ADEPT. The runs comprised five standardized tasks. A posttest visual analog scaled (VAS) questionnaire measuring attitude toward skills testing in general, validation, and performance on ADEPT was used. Subjective responses were compared with objective scores generated through performance on ADEPT. RESULTS: Surgeons emphasize the importance of using a variety of training methods for surgical residents during their residency, including laparoscopic virtual reality simulators. Monitoring of residents' endoscopic progress seemed to be a key issue. Surgeons themselves underestimate their individual performance on ADEPT (mean subjective score of 6.1 vs mean objective score of 6.6). Self-reported performance on ADEPT is unreliable because confidence intervals between the VAS score and the ADEPT score overlap. Surgeons disagree on the validity of ADEPT. The mean score for validity was 5.8, ranging from 0 to 10 with almost equal distribution over the scale. Innate ability is established as surgeons' scores express high concordance between test run and true run, with 72.7% of the participants' true run score within one distance from the test run. CONCLUSIONS: Surgeons cannot correctly predict their standardized individual test result on ADEPT. Performance on ADEPT reflects innate psychomotor ability along with improvement over runs. Surgeons are ambivalent in assessing the validity of ADEPT, irrespective of personal performance.