Amin Madani1,2, Yusuke Watanabe3, Elif Bilgic4, Philip H Pucher5, Melina C Vassiliou6,4, Rajesh Aggarwal6,4,7, Gerald M Fried6,4,7, Elliot J Mitmaker6, Liane S Feldman6,4. 1. Department of Surgery, McGill University, Montreal, Canada. amin.madani@mail.mcgill.ca. 2. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada. amin.madani@mail.mcgill.ca. 3. Department of Gastroenterological Surgery II, Hokkaido University, Sapporo, Japan. 4. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, QC, H3G 1A4, Canada. 5. Department of Surgery and Cancer, Imperial College London, London, UK. 6. Department of Surgery, McGill University, Montreal, Canada. 7. Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Canada.
Abstract
BACKGROUND: Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. METHODS: An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test", VCT). Test-retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform's usability, feasibility and educational value. RESULTS: Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach's α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)]. CONCLUSION: This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.
BACKGROUND: Errors in judgment during laparoscopic cholecystectomy can lead to bile duct injuries and other complications. Despite correlations between outcomes, expertise and advanced cognitive skills, current methods to evaluate these skills remain subjective, rater- and situation-dependent and non-systematic. The purpose of this study was to develop objective metrics using a Web-based platform and to obtain validity evidence for their assessment of decision-making during laparoscopic cholecystectomy. METHODS: An interactive online learning platform was developed ( www.thinklikeasurgeon.com ). Trainees and surgeons from six institutions completed a 12-item assessment, developed based on a cognitive task analysis. Five items required subjects to draw their answer on the surgical field, and accuracy scores were calculated based on an algorithm derived from experts' responses ("visual concordance test", VCT). Test-retest reliability, internal consistency, and correlation with self-reported experience, Global Operative Assessment of Laparoscopic Skills (GOALS) score and Objective Performance Rating Scale (OPRS) score were calculated. Questionnaires were administered to evaluate the platform's usability, feasibility and educational value. RESULTS: Thirty-nine subjects (17 surgeons, 22 trainees) participated. There was high test-retest reliability (intraclass correlation coefficient = 0.95; n = 10) and internal consistency (Cronbach's α = 0.87). The assessment demonstrated significant differences between novices, intermediates and experts in total score (p < 0.01) and VCT score (p < 0.01). There was high correlation between total case number and total score (ρ = 0.83, p < 0.01) and between total case number and VCT (ρ = 0.82, p < 0.01), and moderate to high correlations between total score and GOALS (ρ = 0.66, p = 0.05), VCT and GOALS (ρ = 0.83, p < 0.01), total score and OPRS (ρ = 0.67, p = 0.04), and VCT and OPRS (ρ = 0.78, p = 0.01). Most subjects agreed or strongly agreed that the platform and assessment was easy to use [n = 29 (78 %)], facilitates learning intra-operative decision-making [n = 28 (81 %)], and should be integrated into surgical training [n = 28 (76 %)]. CONCLUSION: This study provides preliminary validity evidence for a novel interactive platform to objectively assess decision-making during laparoscopic cholecystectomy.
Entities:
Keywords:
Assessment; Cholecystectomy; Competency; Education; Performance; Visual concordance test
Authors: Lawrence W Way; Lygia Stewart; Walter Gantert; Kingsway Liu; Crystine M Lee; Karen Whang; John G Hunter Journal: Ann Surg Date: 2003-04 Impact factor: 12.969
Authors: Aslam Ejaz; Gaya Spolverato; Yuhree Kim; Rebecca Dodson; Jason K Sicklick; Henry A Pitt; Keith D Lillemoe; John L Cameron; Timothy M Pawlik Journal: J Am Coll Surg Date: 2014-06-28 Impact factor: 6.113
Authors: David A Cook; Anthony J Levinson; Sarah Garside; Denise M Dupras; Patricia J Erwin; Victor M Montori Journal: JAMA Date: 2008-09-10 Impact factor: 56.272
Authors: Amin Madani; Jordan Gornitsky; Yusuke Watanabe; Cassandre Benay; Maria S Altieri; Philip H Pucher; Roger Tabah; Elliot J Mitmaker Journal: World J Surg Date: 2018-02 Impact factor: 3.352
Authors: Joshua Weis; L Michael Brunt; Amin Madani; Dana Telem; Madhuri Nagaraj; Horacio Asbun; Brian R Davis; Sharmila Dissanaike; Michael B Ujiki; Carl J Westcott; Adnan Alseidi Journal: Surg Endosc Date: 2022-08-25 Impact factor: 3.453
Authors: Hamzeh Naghawi; Johnny Chau; Amin Madani; Pepa Kaneva; John Monson; Carmen Mueller; Lawrence Lee Journal: Tech Coloproctol Date: 2022-05-03 Impact factor: 3.699
Authors: Ingrid A Woelfel; Brentley Q Smith; Ritu Salani; Alan E Harzman; Amalia L Cochran; Xiaodong Phoenix Chen Journal: Am J Surg Date: 2021-03-18 Impact factor: 3.125
Authors: Amin Madani; Babak Namazi; Maria S Altieri; Daniel A Hashimoto; Angela Maria Rivera; Philip H Pucher; Allison Navarrete-Welton; Ganesh Sankaranarayanan; L Michael Brunt; Allan Okrainec; Adnan Alseidi Journal: Ann Surg Date: 2020-11-13 Impact factor: 13.787