Yusuke Watanabe1,2, Elif Bilgic3, Ekaterina Lebedeva4, Katherine M McKendy3, Liane S Feldman3, Gerald M Fried3, Melina C Vassiliou5. 1. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 316, Montreal, QC, H3G 1A4, Canada. ywatanabe328@gmail.com. 2. Department of Gastroenterological Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Hokkaido, Japan. ywatanabe328@gmail.com. 3. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 316, Montreal, QC, H3G 1A4, Canada. 4. The Henry K.M. De Kuyper Education Centre, McGill University Health Centre, Montreal, QC, Canada. 5. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1650, Cedar Avenue, L9. 316, Montreal, QC, H3G 1A4, Canada. melina.vassiliou@mcgill.ca.
Abstract
BACKGROUND: Multiple tools are available to assess clinical performance of laparoscopic cholecystectomy (LC), but there are no guidelines on how best to implement and interpret them in educational settings. The purpose of this systematic review was to identify and critically appraise LC assessment tools and their measurement properties, in order to make recommendations for their implementation in surgical training. METHODS: A systematic search (1989-2013) was conducted in MEDLINE, Embase, Scopus, Cochrane, and grey literature sources. Evidence for validity (content, response process, internal structure, relations to other variables, and consequences) and the conditions in which the evidence was obtained were evaluated. RESULTS: A total of 54 articles were included for qualitative synthesis. Fifteen technical skills and two non-technical skills assessment tools were identified. The 17 tools were used for either: recorded procedures (nine tools, 60%), direct observation (five tools, 30%), or both (three tools, 18%). Fourteen (82%) tools reported inter-rater reliability and one reported a Generalizability Theory coefficient. Nine (53%) had evidence for validity based on clinical experience and 11 (65%) compared scores to other assessments. Consequences of scores, educational impact, applications to residency training, and how raters were trained were not clearly reported. No studies mentioned cost. CONCLUSIONS: The most commonly reported validity evidence was inter-rater reliability and relationships to other known variables. Consequences of assessments and rater training were not clearly reported. These data and the evidence for validity should be taken into consideration when deciding how to select and implement a tool to assess performance of LC, and especially how to interpret the results.
BACKGROUND: Multiple tools are available to assess clinical performance of laparoscopic cholecystectomy (LC), but there are no guidelines on how best to implement and interpret them in educational settings. The purpose of this systematic review was to identify and critically appraise LC assessment tools and their measurement properties, in order to make recommendations for their implementation in surgical training. METHODS: A systematic search (1989-2013) was conducted in MEDLINE, Embase, Scopus, Cochrane, and grey literature sources. Evidence for validity (content, response process, internal structure, relations to other variables, and consequences) and the conditions in which the evidence was obtained were evaluated. RESULTS: A total of 54 articles were included for qualitative synthesis. Fifteen technical skills and two non-technical skills assessment tools were identified. The 17 tools were used for either: recorded procedures (nine tools, 60%), direct observation (five tools, 30%), or both (three tools, 18%). Fourteen (82%) tools reported inter-rater reliability and one reported a Generalizability Theory coefficient. Nine (53%) had evidence for validity based on clinical experience and 11 (65%) compared scores to other assessments. Consequences of scores, educational impact, applications to residency training, and how raters were trained were not clearly reported. No studies mentioned cost. CONCLUSIONS: The most commonly reported validity evidence was inter-rater reliability and relationships to other known variables. Consequences of assessments and rater training were not clearly reported. These data and the evidence for validity should be taken into consideration when deciding how to select and implement a tool to assess performance of LC, and especially how to interpret the results.
Authors: Neal E Seymour; Anthony G Gallagher; Sanziana A Roman; Michael K O'Brien; Vipin K Bansal; Dana K Andersen; Richard M Satava Journal: Ann Surg Date: 2002-10 Impact factor: 12.969
Authors: Gunnar Ahlberg; Lars Enochsson; Anthony G Gallagher; Leif Hedman; Christian Hogman; David A McClusky; Stig Ramel; C Daniel Smith; Dag Arvidsson Journal: Am J Surg Date: 2007-06 Impact factor: 2.565
Authors: Marco von Strauss Und Torney; Sohelia Aghlmandi; Jasmin Zeindler; Debora Nowakowski; Christian A Nebiker; Christoph Kettelhack; Rachel Rosenthal; Raoul A Droeser; Savas D Soysal; Henry Hoffmann; Robert Mechera Journal: Surg Endosc Date: 2018-05-21 Impact factor: 4.584
Authors: Saba Balvardi; Anitha Kammili; Melissa Hanson; Carmen Mueller; Melina Vassiliou; Lawrence Lee; Kevin Schwartzman; Julio F Fiore; Liane S Feldman Journal: Surg Endosc Date: 2022-05-12 Impact factor: 4.584
Authors: Mona W Schmidt; Caelan M Haney; Karl-Friedrich Kowalewski; Vasile V Bintintan; Mohammed Abu Hilal; Alberto Arezzo; Marcus Bahra; Marc G Besselink; Matthias Biebl; Luigi Boni; Michele Diana; Jan H Egberts; Lars Fischer; Nader Francis; Daniel A Hashimoto; Daniel Perez; Marlies Schijven; Moritz Schmelzle; Marek Soltes; Lee Swanstrom; Thilo Welsch; Beat P Müller-Stich; Felix Nickel Journal: Surg Endosc Date: 2021-11-09 Impact factor: 3.453
Authors: Saba Balvardi; Koorosh Semsar-Kazerooni; Pepa Kaneva; Carmen Mueller; Melina Vassiliou; Mohammed Al Mahroos; Julio F Fiore; Kevin Schwartzman; Liane S Feldman Journal: Surg Endosc Date: 2022-08-03 Impact factor: 3.453
Authors: Gemma Humm; Helen Mohan; Christina Fleming; Rhiannon Harries; Christopher Wood; Khaled Dawas; Danail Stoyanov; Laurence B Lovat Journal: BJS Open Date: 2022-07-07