Heinrich Husslein1, Lindsay Shirreff2, Eliane M Shore2, Guylaine G Lefebvre2, Teodor P Grantcharov3. 1. Division of Obstetrics and Gynaecology, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada; Department of Obstetrics and Gynecology, Medical University Vienna, Vienna, Austria. Electronic address: heinrich.husslein@meduniwien.ac.at. 2. Division of Obstetrics and Gynaecology, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada. 3. Division of General Surgery, St. Michael's Hospital, University of Toronto, Toronto, Ontario, Canada.
Abstract
OBJECTIVE: Global rating scales are commonly used to rate surgeons' skill level. However, these tools lack granularity required for specific skill feedback. Recently, an alternative framework has been developed that is designed to measure technical errors during laparoscopy. The purpose of the present study was to gather validity evidence for the Generic Error Rating Tool (GERT) in gynecologic laparoscopy. DESIGN: Video recordings of total laparoscopic hysterectomies were analyzed by 2 blinded reviewers using the GERT and the Objective Structured Assessment of Technical Skills (OSATS) scale. Several sources of validity were examined according to the unitary framework of validity. Main outcomes were interrater and intrarater reliability regarding total number of errors and events. Further, surgeons were grouped according to OSATS scores (OSATS ≥ 28 = high performers and OSATS < 28 = low performers), and the number of errors and events was compared between groups. Correlation analysis between GERT and OSATS scores was performed. Lastly, error distribution within procedure steps was explored and compared between high- and low-performing surgeons. SETTING: University teaching hospital. PARTICIPANTS: A total of 20 anonymized video recordings of total laparoscopic hysterectomies. RESULTS: Interrater and intrarater reliability was high (intraclass correlation coefficient >0.95) for total number of errors and events. Low performers made significantly more errors than high performers did (median = 49.5 [interquartile range: 34.5-66] vs median = 31 [interquartile range: 16.75-35.25], p = 0.002). There was a significant negative correlation between individual OSATS scores and total number of errors (Spearman ρ = -0.76, p < 0.001, and ρ = -0.88, p < 0.001, for raters 1 and 2, respectively). Error distribution varied between operative steps, and low performers made more errors in some steps, but not in others. CONCLUSION: GERT allows for objective and reproducible assessment of technical errors during gynecologic laparoscopy and could be used for performance analysis and personalized surgical education and training.
OBJECTIVE: Global rating scales are commonly used to rate surgeons' skill level. However, these tools lack granularity required for specific skill feedback. Recently, an alternative framework has been developed that is designed to measure technical errors during laparoscopy. The purpose of the present study was to gather validity evidence for the Generic Error Rating Tool (GERT) in gynecologic laparoscopy. DESIGN: Video recordings of total laparoscopic hysterectomies were analyzed by 2 blinded reviewers using the GERT and the Objective Structured Assessment of Technical Skills (OSATS) scale. Several sources of validity were examined according to the unitary framework of validity. Main outcomes were interrater and intrarater reliability regarding total number of errors and events. Further, surgeons were grouped according to OSATS scores (OSATS ≥ 28 = high performers and OSATS < 28 = low performers), and the number of errors and events was compared between groups. Correlation analysis between GERT and OSATS scores was performed. Lastly, error distribution within procedure steps was explored and compared between high- and low-performing surgeons. SETTING: University teaching hospital. PARTICIPANTS: A total of 20 anonymized video recordings of total laparoscopic hysterectomies. RESULTS: Interrater and intrarater reliability was high (intraclass correlation coefficient >0.95) for total number of errors and events. Low performers made significantly more errors than high performers did (median = 49.5 [interquartile range: 34.5-66] vs median = 31 [interquartile range: 16.75-35.25], p = 0.002). There was a significant negative correlation between individual OSATS scores and total number of errors (Spearman ρ = -0.76, p < 0.001, and ρ = -0.88, p < 0.001, for raters 1 and 2, respectively). Error distribution varied between operative steps, and low performers made more errors in some steps, but not in others. CONCLUSION: GERT allows for objective and reproducible assessment of technical errors during gynecologic laparoscopy and could be used for performance analysis and personalized surgical education and training.
Authors: Ryan Daniel; Tyler McKechnie; Colin C Kruse; Marc Levin; Yung Lee; Aristithes G Doumouras; Dennis Hong; Cagla Eskicioglu Journal: Surg Endosc Date: 2022-06-23 Impact factor: 3.453
Authors: Eyüp Murat Yılmaz; Aykut Soyder; Murat Aksu; Ali Doğan Bozdağ; Şükrü Boylu; Akay Edizsoy; Şadi Ballı; Mustafa Agah Tekindal Journal: Turk J Surg Date: 2017-12-01
Authors: Alexander J W Beulens; Willem M Brinkman; Evert L Koldewijn; Ad J M Hendrikx; Jean Paul A van Basten; Jeroen J G van Merriënboer; Henk G Van der Poel; Chris H Bangma; Cordula Wagner Journal: Eur Urol Open Sci Date: 2020-07-03