Hamzeh Naghawi1, Johnny Chau1, Amin Madani2, Pepa Kaneva1, John Monson3, Carmen Mueller1, Lawrence Lee4,5. 1. The Steinberg-Bernstein Centre for Minimally Invasive Surgery, McGill University Health Center, Montreal, QC, Canada. 2. The University Health Network - Toronto General Hospital, Toronto, ON, Canada. 3. AdventHealth Medical Group, Orlando, FL, USA. 4. The Steinberg-Bernstein Centre for Minimally Invasive Surgery, McGill University Health Center, Montreal, QC, Canada. larry.lee@mcgill.ca. 5. Colon and Rectal Surgery, Department of Surgery, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada. larry.lee@mcgill.ca.
Abstract
BACKGROUND: Transanal total mesorectal excision (TATME) is difficult to learn and can result in serious complications. Current paradigms for assessing performance and competency may be insufficient. This study aims to develop and provide preliminary validity evidence for a TATME virtual assessment tool (TATME-VAT) to assess the cognitive skills necessary to safely complete TATME dissection. METHODS: Participants from North America, Europe, Japan and China completed the test via an interactive online platform between 11/2019 and 05/2020. They were grouped into expert, experienced and novice surgeons depending on the number of independently performed TATMEs. TATME-VAT is a 24-item web-based assessment evaluating advanced cognitive skills, designed according to a blueprint from consensus guidelines. Eight items were multiple choice questions. Sixteen items required making annotations on still frames of TATME videos (VCT) and were scored using a validated algorithm derived from experts' responses. Annotation (range 0-100), multiple choice (range 0-100), and overall scores (sum of annotation and multiple-choice scores, normalized to μ = 50 and σ = 10) were reported. RESULTS: There were significant differences between the expert, experienced, and novice groups for the annotation (p < 0.001), multiple-choice (p < 0.001), and overall scores (p < 0.001). The annotation (p = 0.439) and overall (p = 0.152) scores were similar between the experienced and novice groups. Annotation scores were higher in participants with 51 or more vs. 30-50 vs. less than 30 cases. Scores were also lower in users with a self-reported recent complication vs. those without. CONCLUSIONS: This study describes the development of an interactive video-based virtual assessment tool for TATME dissection and provides initial validity evidence for its use.
BACKGROUND: Transanal total mesorectal excision (TATME) is difficult to learn and can result in serious complications. Current paradigms for assessing performance and competency may be insufficient. This study aims to develop and provide preliminary validity evidence for a TATME virtual assessment tool (TATME-VAT) to assess the cognitive skills necessary to safely complete TATME dissection. METHODS: Participants from North America, Europe, Japan and China completed the test via an interactive online platform between 11/2019 and 05/2020. They were grouped into expert, experienced and novice surgeons depending on the number of independently performed TATMEs. TATME-VAT is a 24-item web-based assessment evaluating advanced cognitive skills, designed according to a blueprint from consensus guidelines. Eight items were multiple choice questions. Sixteen items required making annotations on still frames of TATME videos (VCT) and were scored using a validated algorithm derived from experts' responses. Annotation (range 0-100), multiple choice (range 0-100), and overall scores (sum of annotation and multiple-choice scores, normalized to μ = 50 and σ = 10) were reported. RESULTS: There were significant differences between the expert, experienced, and novice groups for the annotation (p < 0.001), multiple-choice (p < 0.001), and overall scores (p < 0.001). The annotation (p = 0.439) and overall (p = 0.152) scores were similar between the experienced and novice groups. Annotation scores were higher in participants with 51 or more vs. 30-50 vs. less than 30 cases. Scores were also lower in users with a self-reported recent complication vs. those without. CONCLUSIONS: This study describes the development of an interactive video-based virtual assessment tool for TATME dissection and provides initial validity evidence for its use.
Authors: Marta Penna; Roel Hompes; Steve Arnold; Greg Wynn; Ralph Austin; Janindra Warusavitarne; Brendan Moran; George B Hanna; Neil J Mortensen; Paris P Tekkis Journal: Ann Surg Date: 2017-07 Impact factor: 12.969
Authors: Aaron S Rickles; David W Dietz; George J Chang; Steven D Wexner; Mariana E Berho; Feza H Remzi; Frederick L Greene; James W Fleshman; Maher A Abbas; Walter Peters; Katia Noyes; John R T Monson; Fergal J Fleming Journal: Ann Surg Date: 2015-12 Impact factor: 12.969
Authors: Martijn Hgm van der Pas; Eva Haglind; Miguel A Cuesta; Alois Fürst; Antonio M Lacy; Wim Cj Hop; Hendrik Jaap Bonjer Journal: Lancet Oncol Date: 2013-02-06 Impact factor: 41.316
Authors: James Fleshman; Megan Branda; Daniel J Sargent; Anne Marie Boller; Virgilio George; Maher Abbas; Walter R Peters; Dipen Maun; George Chang; Alan Herline; Alessandro Fichera; Matthew Mutch; Steven Wexner; Mark Whiteford; John Marks; Elisa Birnbaum; David Margolin; David Larson; Peter Marcello; Mitchell Posner; Thomas Read; John Monson; Sherry M Wren; Peter W T Pisters; Heidi Nelson Journal: JAMA Date: 2015-10-06 Impact factor: 56.272
Authors: Andrew R L Stevenson; Michael J Solomon; John W Lumley; Peter Hewett; Andrew D Clouston; Val J Gebski; Lucy Davies; Kate Wilson; Wendy Hague; John Simes Journal: JAMA Date: 2015-10-06 Impact factor: 56.272
Authors: N S Fearnhead; A G Acheson; S R Brown; L Hancock; A Harikrishnan; S B Kelly; C A Maxwell-Armstrong; P M Sagar; S Siddiqi; C J Walsh; J M D Wheeler; J F Abercrombie Journal: Colorectal Dis Date: 2020-07 Impact factor: 3.788
Authors: S E van Oostendorp; H J Belgers; B T Bootsma; J C Hol; E J T H Belt; W Bleeker; F C Den Boer; A Demirkiran; M S Dunker; H F J Fabry; E J R Graaf; J J Knol; S J Oosterling; G D Slooter; D J A Sonneveld; A K Talsma; H L Van Westreenen; M Kusters; R Hompes; H J Bonjer; C Sietses; J B Tuynman Journal: Br J Surg Date: 2020-04-04 Impact factor: 6.939