Shanley B Deal1, Dimitrios Stefanidis2, Dana Telem3, Robert D Fanelli4, Marian McDonald5, Michael Ujiki6, L Michael Brunt7, Adnan A Alseidi8. 1. Graduate Medical Education, Virginia Mason Medical Center, Mailstop H8-GME, 1100 9th Ave., Seattle, WA, 98101, USA. Shanley.Deal@virginiamason.org. 2. Department of Surgery, Indiana University, Bloomington, IN, USA. 3. Department of Surgery, University of Michigan, Ann Arbor, MI, USA. 4. Department of Surgery, The Guthrie Clinic, Sayre, PA, USA. 5. General Surgery, St. Luke's University Health Network, Bethlehem, PA, USA. 6. General Surgery, NorthShore University HealthSystem, Evanston, IL, USA. 7. Department of Surgery, Washington University St. Louis, St. Louis, MO, USA. 8. Department of Surgery, Virginia Mason Medical Center, Seattle, WA, USA.
Abstract
BACKGROUND: Faculty experts (FE) and crowd workers (CW) can assess technical skill, but assessment of operative technique has not been explored. We sought to evaluate if CW could be taught to assess completion of the critical view of safety (CVS) in laparoscopic cholecystectomy. METHODS: We prepared 160 blinded, surgical videos of laparoscopic cholecystectomy from public domain websites. Videos were edited to ≤60 s, ending when a structure was cut/clipped. CW analyzed videos using Global Objective Assessment of Laparoscopic Skills (GOALS) and CVS criteria assessment tools after watching an instructional tutorial. Ten videos were randomly selected from each performance quartile based on GOALS. Five FE rated the 40 videos using GOALS and CVS. Linear mixed effects models derived average CW and FE ratings for GOALS and CVS for each video. Spearman correlation coefficients (SCC) were used to assess the degree of correlation between performance measures. Satisfactory completion of the CVS was defined as scoring an average CVS ≥ 5. Videos with an average GOALS ≥ 15 were considered top technical performers. RESULTS: A high degree of correlation was seen between all performance measures: CVS ratings between CW and FE, SCC 0.89 (p < 0.001); GOALS and CVS ratings SCC 0.77 (p < 0.001) for CW, and SCC 0.71 (p < 0.001) for FE. Sixteen videos were assigned top technical performer ratings by both CW and FE but the average CVS was inadequate (3.8 and 3.6, respectively), and the percentage of satisfactory CVS ≥ 5 was 12.5%. CONCLUSIONS: A high degree of correlation was found between CW and FE in assessment of the CVS. However, in this video analysis, high technical performers did not achieve a complete CVS in most cases. Educating CW to assess operative technique for the identification of low or average performers is feasible and may broaden the application of this assessment and feedback tool.
BACKGROUND: Faculty experts (FE) and crowd workers (CW) can assess technical skill, but assessment of operative technique has not been explored. We sought to evaluate if CW could be taught to assess completion of the critical view of safety (CVS) in laparoscopic cholecystectomy. METHODS: We prepared 160 blinded, surgical videos of laparoscopic cholecystectomy from public domain websites. Videos were edited to ≤60 s, ending when a structure was cut/clipped. CW analyzed videos using Global Objective Assessment of Laparoscopic Skills (GOALS) and CVS criteria assessment tools after watching an instructional tutorial. Ten videos were randomly selected from each performance quartile based on GOALS. Five FE rated the 40 videos using GOALS and CVS. Linear mixed effects models derived average CW and FE ratings for GOALS and CVS for each video. Spearman correlation coefficients (SCC) were used to assess the degree of correlation between performance measures. Satisfactory completion of the CVS was defined as scoring an average CVS ≥ 5. Videos with an average GOALS ≥ 15 were considered top technical performers. RESULTS: A high degree of correlation was seen between all performance measures: CVS ratings between CW and FE, SCC 0.89 (p < 0.001); GOALS and CVS ratings SCC 0.77 (p < 0.001) for CW, and SCC 0.71 (p < 0.001) for FE. Sixteen videos were assigned top technical performer ratings by both CW and FE but the average CVS was inadequate (3.8 and 3.6, respectively), and the percentage of satisfactory CVS ≥ 5 was 12.5%. CONCLUSIONS: A high degree of correlation was found between CW and FE in assessment of the CVS. However, in this video analysis, high technical performers did not achieve a complete CVS in most cases. Educating CW to assess operative technique for the identification of low or average performers is feasible and may broaden the application of this assessment and feedback tool.
Keywords:
Critical view of safety; Crowd-sourced assessment of technical skill; GOALS; Laparoscopic cholecystectomy; Video-based education
Authors: Crystal B Chen; Francesco Palazzo; Stephen M Doane; Jordan M Winter; Harish Lavu; Karen A Chojnacki; Ernest L Rosato; Charles J Yeo; Michael J Pucci Journal: Surg Endosc Date: 2016-08-05 Impact factor: 4.584
Authors: K T Buddingh; A N Morks; H O ten Cate Hoedemaker; C B Blaauw; G M van Dam; R J Ploeg; H S Hofker; V B Nieuwenhuijs Journal: Surg Endosc Date: 2011-07-27 Impact factor: 4.584
Authors: William B Lyman; Michael J Passeri; Keith Murphy; Imran A Siddiqui; Adeel S Khan; David A Iannitti; John B Martinie; Erin H Baker; Dionisios Vrochides Journal: Surg Endosc Date: 2020-06-16 Impact factor: 4.584
Authors: Joshua Weis; L Michael Brunt; Amin Madani; Dana Telem; Madhuri Nagaraj; Horacio Asbun; Brian R Davis; Sharmila Dissanaike; Michael B Ujiki; Carl J Westcott; Adnan Alseidi Journal: Surg Endosc Date: 2022-08-25 Impact factor: 3.453
Authors: Grace L Paley; Rebecca Grove; Tejas C Sekhar; Jack Pruett; Michael V Stock; Tony N Pira; Steven M Shields; Evan L Waxman; Bradley S Wilson; Mae O Gordon; Susan M Culican Journal: J Surg Educ Date: 2021-02-25 Impact factor: 2.891
Authors: Mihai-Calin Pavel; Mar Achalandabaso Boira; Yasir Bashir; Robert Memba; Erik Llácer; Laia Estalella; Elisabeth Julià; Kevin C Conlon; Rosa Jorba Journal: Syst Rev Date: 2022-03-03