Allan Okrainec1,2,3, Melina Vassiliou4, M Carolina Jimenez5, Oscar Henao6, Pepa Kaneva4, E Matt Ritter7. 1. Temerty/Chang Telesimulation Centre, University Health Network, Toronto, ON, Canada. allan.okrainec@uhn.ca. 2. Department of Surgery, University of Toronto, Toronto, ON, Canada. allan.okrainec@uhn.ca. 3. Division of General Surgery, University Health Network, Main Pavilion, 8th Floor, Suite 325, 399 Bathurst Street, Toronto, ON, M5T 2S8, Canada. allan.okrainec@uhn.ca. 4. Montreal General Hospital, McGill University, Montreal, QC, Canada. 5. Temerty/Chang Telesimulation Centre, University Health Network, Toronto, ON, Canada. 6. Department of Surgery, University of Toronto, Toronto, ON, Canada. 7. Uniformed Services University, Bethesda, MD, USA.
Abstract
INTRODUCTION: Maintaining the existing FLS test centers requires considerable investment in human and financial resources. It can also be particularly challenging for those outside of North America to become certified due to the limited number of international test centers. Preliminary work suggests that it is possible to reliably score the FLS manual skills component remotely using low-cost videoconferencing technology. Significant work remains to ensure that testing procedures adhere to standards defined by SAGES for this approach to be considered equivalent to standard on-site testing. OBJECTIVE: To validate the integrity and validity of the FLS manual skills examination administered remotely in a real-world environment according to FLS testing protocols and to evaluate participants' experience with the setting. METHODS: Individuals with various levels of training from the University of Toronto completed a pre- and a post-test questionnaire. Participants presented to one of the two FLS testing rooms available for the study, each connected via Skype to a separate room with a FLS proctor who administered and scored the test remotely (RP). An on-site proctor (OP) was present in the room as a control. An invigilator was also present in the testing room to follow directions from the RP and ensure the integrity of test materials. RESULTS: Twenty-one participants were recruited, and 20 completed the test. There was no significant difference between scores by RP and OP. Interrater reliability between the RP and OP was excellent. One critical error was missed by the RP, but this would not have affected the test outcome. Participants reported being highly satisfied. CONCLUSION: We demonstrate that proctors located remotely can administer the FLS skills test in a secure and reliable fashion, with excellent interrater reliability compared to an on-site proctor. Remote proctoring of the FLS examination could become a strategy to increase certification rates while containing costs.
INTRODUCTION: Maintaining the existing FLS test centers requires considerable investment in human and financial resources. It can also be particularly challenging for those outside of North America to become certified due to the limited number of international test centers. Preliminary work suggests that it is possible to reliably score the FLS manual skills component remotely using low-cost videoconferencing technology. Significant work remains to ensure that testing procedures adhere to standards defined by SAGES for this approach to be considered equivalent to standard on-site testing. OBJECTIVE: To validate the integrity and validity of the FLS manual skills examination administered remotely in a real-world environment according to FLS testing protocols and to evaluate participants' experience with the setting. METHODS: Individuals with various levels of training from the University of Toronto completed a pre- and a post-test questionnaire. Participants presented to one of the two FLS testing rooms available for the study, each connected via Skype to a separate room with a FLS proctor who administered and scored the test remotely (RP). An on-site proctor (OP) was present in the room as a control. An invigilator was also present in the testing room to follow directions from the RP and ensure the integrity of test materials. RESULTS: Twenty-one participants were recruited, and 20 completed the test. There was no significant difference between scores by RP and OP. Interrater reliability between the RP and OP was excellent. One critical error was missed by the RP, but this would not have affected the test outcome. Participants reported being highly satisfied. CONCLUSION: We demonstrate that proctors located remotely can administer the FLS skills test in a secure and reliable fashion, with excellent interrater reliability compared to an on-site proctor. Remote proctoring of the FLS examination could become a strategy to increase certification rates while containing costs.
Entities:
Keywords:
Fundamentals of laparoscopic surgery; Medical education; Minimally invasive surgery
Authors: Jeffrey H Peters; Gerald M Fried; Lee L Swanstrom; Nathaniel J Soper; Lelan F Sillin; Bruce Schirmer; Kaaren Hoffman Journal: Surgery Date: 2004-01 Impact factor: 3.982
Authors: Gerald M Fried; Liane S Feldman; Melina C Vassiliou; Shannon A Fraser; Donna Stanbridge; Gabriela Ghitulescu; Christopher G Andrew Journal: Ann Surg Date: 2004-09 Impact factor: 12.969
Authors: M C Vassiliou; G A Ghitulescu; L S Feldman; D Stanbridge; K Leffondré; H H Sigman; G M Fried Journal: Surg Endosc Date: 2006-02-27 Impact factor: 4.584
Authors: Daniel J Scott; E Matt Ritter; Seifu T Tesfay; Elisabeth A Pimentel; Alykhan Nagji; Gerald M Fried Journal: Surg Endosc Date: 2008-02-13 Impact factor: 4.584