Bradley J Champagne1, Scott R Steele, Samantha K Hendren, Paul M Bakaki, Patricia L Roberts, Conor P Delaney, Justin T Brady, Helen M MacRae. 1. 1 Department of Surgery, Cleveland Clinic Fairview Hospital, Cleveland, Ohio 2 Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, Ohio 3 Department of Surgery, University of Michigan, Ann Arbor, Michigan 4 Department of Epidemiology and Biostatistics, School of Medicine, Case Western Reserve University, Cleveland, Ohio 5 Department of Surgery, Lahey Clinic, Tufts University School of Medicine, Burlington, Massachusetts 6 Digestive Disease and Surgery Institute, Cleveland Clinic Foundation, Cleveland, Ohio 7 Department of Surgery, University of Toronto, Division of General Surgery, Mount Sinai Hospital, Toronto, Ontario, Canada.
Abstract
BACKGROUND: The lack of consensus for performance assessment of laparoscopic colorectal resection is a major impediment to quality improvement. OBJECTIVE: The purpose of this study was to develop and assess the validity of an evaluation tool for laparoscopic colectomy that is feasible for wide implementation. DESIGN: During the pilot phase, a small group of experts modified previous assessment tools by watching videos for laparoscopic right colectomy with the following categories of experience: novice (less than 20 cases), intermediate (50-100 cases), and expert (more than 500 cases). After achieving sufficient reliability (κ > 0.8), a user-friendly tool was validated among a large group of blinded, trained experts. SETTING: The study was conducted through the American Society of Colon and Rectal Surgeons Operative Competency Evaluation Committee. PATIENTS: Raters were from the Operative Competency Evaluation Committee of the American Society of Colon and Rectal Surgeons. MAIN OUTCOME MEASURES: Assessment tool reliability and internal consistency were measured. RESULTS: From October 2014 through February 2015, 4 groups of 5 raters blinded to surgeon skill level evaluated 6 different laparoscopic right colectomy videos (novice = 2, intermediate = 2, expert = 2). The overall Cronbach α was 0.98 (>0.9 = excellent internal consistency). The intraclass correlation for the overall assessment was 0.93 (range, 0.77-0.93) and was >0.74 (excellent) for each step. The average scores (scale, 1-5) for experts were significantly better than those in the intermediate category, with a mean (SD) of 4.51 (0.56) versus 2.94 (0.56; p = 0.003). Videos in the intermediate group scored more favorably than beginner videos for each individual step and overall performance (mean (SD) = 3.00 (0.32) vs 1.78 (0.42); p = 0.006). LIMITATIONS: The study was limited by rater bias to technique and style. CONCLUSIONS: The unique and robust methodology in this trial produced an assessment tool that was feasible for raters to use when assessing videotaped laparoscopic right hemicolectomies. The potential applications for this new tool are widespread, including both training and evaluation of competence at the attending level. See Video Abstract at http://links.lww.com/DCR/A369, http://links.lww.com/DCR/A370, http://links.lww.com/DCR/A371.
BACKGROUND: The lack of consensus for performance assessment of laparoscopic colorectal resection is a major impediment to quality improvement. OBJECTIVE: The purpose of this study was to develop and assess the validity of an evaluation tool for laparoscopic colectomy that is feasible for wide implementation. DESIGN: During the pilot phase, a small group of experts modified previous assessment tools by watching videos for laparoscopic right colectomy with the following categories of experience: novice (less than 20 cases), intermediate (50-100 cases), and expert (more than 500 cases). After achieving sufficient reliability (κ > 0.8), a user-friendly tool was validated among a large group of blinded, trained experts. SETTING: The study was conducted through the American Society of Colon and Rectal Surgeons Operative Competency Evaluation Committee. PATIENTS: Raters were from the Operative Competency Evaluation Committee of the American Society of Colon and Rectal Surgeons. MAIN OUTCOME MEASURES: Assessment tool reliability and internal consistency were measured. RESULTS: From October 2014 through February 2015, 4 groups of 5 raters blinded to surgeon skill level evaluated 6 different laparoscopic right colectomy videos (novice = 2, intermediate = 2, expert = 2). The overall Cronbach α was 0.98 (>0.9 = excellent internal consistency). The intraclass correlation for the overall assessment was 0.93 (range, 0.77-0.93) and was >0.74 (excellent) for each step. The average scores (scale, 1-5) for experts were significantly better than those in the intermediate category, with a mean (SD) of 4.51 (0.56) versus 2.94 (0.56; p = 0.003). Videos in the intermediate group scored more favorably than beginner videos for each individual step and overall performance (mean (SD) = 3.00 (0.32) vs 1.78 (0.42); p = 0.006). LIMITATIONS: The study was limited by rater bias to technique and style. CONCLUSIONS: The unique and robust methodology in this trial produced an assessment tool that was feasible for raters to use when assessing videotaped laparoscopic right hemicolectomies. The potential applications for this new tool are widespread, including both training and evaluation of competence at the attending level. See Video Abstract at http://links.lww.com/DCR/A369, http://links.lww.com/DCR/A370, http://links.lww.com/DCR/A371.
Authors: Kyle Lam; Junhong Chen; Zeyu Wang; Fahad M Iqbal; Ara Darzi; Benny Lo; Sanjay Purkayastha; James M Kinross Journal: NPJ Digit Med Date: 2022-03-03