BACKGROUND: Direct observation with structured criteria for performance is the most reliable and valid method of assessing technical skill during operative procedures. We developed such a system to evaluate technical performance during a laparoscopic cholecystectomy. The reliability and validity of the system were tested by analyzing the correlation among three observers in a multicenter study and comparing performance with years of surgical experience. STUDY DESIGN: Thirty consecutive cases of laparoscopic cholecystectomy were recorded on videotape, 10 from each of 3 institutions. Independent scores were generated by three observers examining each of the videotapes, providing a total of 90 scores. Points were awarded for successful completion of each of 23 different steps required to perform a laparoscopic cholecystectomy. Error points were tabulated based on the frequency and relative severity of each of 21 potential technical mistakes during the operation. The final score was assumed to be a relative measure of technical skill and was derived by subtracting error points from points awarded for completion of each step of the procedure. Pearson correlation coefficients were used to assess agreement among examiners and correlation with year of surgical experience. RESULTS: Agreement in final scores among the three observers was excellent (r = 0.74-0.96) despite the fact that one observer assigned significantly fewer error points. Correlation between year of experience and two-handed technique scoring was good (r = 0.5, p = 0.057), but the correlation between experience and one-handed technique scores was poor (r = 0.02). CONCLUSIONS: The technical skills required to perform laparoscopic cholecystectomy can reliably be measured using this tool. This method can be used to track the learning curve of surgeons in training, evaluate the efficacy of alternative training tools, and provide a means of self-assessment for the trainee.
BACKGROUND: Direct observation with structured criteria for performance is the most reliable and valid method of assessing technical skill during operative procedures. We developed such a system to evaluate technical performance during a laparoscopic cholecystectomy. The reliability and validity of the system were tested by analyzing the correlation among three observers in a multicenter study and comparing performance with years of surgical experience. STUDY DESIGN: Thirty consecutive cases of laparoscopic cholecystectomy were recorded on videotape, 10 from each of 3 institutions. Independent scores were generated by three observers examining each of the videotapes, providing a total of 90 scores. Points were awarded for successful completion of each of 23 different steps required to perform a laparoscopic cholecystectomy. Error points were tabulated based on the frequency and relative severity of each of 21 potential technical mistakes during the operation. The final score was assumed to be a relative measure of technical skill and was derived by subtracting error points from points awarded for completion of each step of the procedure. Pearson correlation coefficients were used to assess agreement among examiners and correlation with year of surgical experience. RESULTS: Agreement in final scores among the three observers was excellent (r = 0.74-0.96) despite the fact that one observer assigned significantly fewer error points. Correlation between year of experience and two-handed technique scoring was good (r = 0.5, p = 0.057), but the correlation between experience and one-handed technique scores was poor (r = 0.02). CONCLUSIONS: The technical skills required to perform laparoscopic cholecystectomy can reliably be measured using this tool. This method can be used to track the learning curve of surgeons in training, evaluate the efficacy of alternative training tools, and provide a means of self-assessment for the trainee.
Authors: Neal E Seymour; Anthony G Gallagher; Sanziana A Roman; Michael K O'Brien; Vipin K Bansal; Dana K Andersen; Richard M Satava Journal: Ann Surg Date: 2002-10 Impact factor: 12.969