Catalina Ortiz1, Francisca Belmar1, Rolando Rebolledo2, Javier Vela1, Caterina Contreras1, Martin Inzunza1, Juan Pablo Ramos3, Analía Zinco3, Adnan Alseidi4, Julián Varas1, Nicolás Jarufe1, Pablo Achurra5. 1. Experimental Surgery and Simulation Center, Department of Digestive Surgery. Pontificia Universidad Católica de Chile, Santiago, Chile. 2. Institute for Biomedical Engineering, Pontificia Universidad Católica de Chile, Santiago, Chile. 3. Trauma Surgery and Emergency Medicine Department, Hospital Dr. Sotero del Río, Santiago, Chile. 4. Department of Surgery, University of California San Francisco, San Francisco, California. 5. Experimental Surgery and Simulation Center, Department of Digestive Surgery. Pontificia Universidad Católica de Chile, Santiago, Chile. Electronic address: achurrapablo@gmail.com.
Abstract
BACKGROUND: Trauma is one of the main causes of death globally, and appropriate surgical care is crucial to impact mortality. However, resident-performed trauma cases have diminished in the last 10 years. Simulation-based tools have proven to be effective to evaluate practical skills in a variety of settings. However, there is a lack of evidence regarding proper validation of trauma surgery models. OBJECTIVE: The aim of this study was to evaluate under a contemporary validity framework, an objective structured clinical evaluation (OSCE) scenario for the assessment of basic and advanced surgical skills in trauma and emergency surgery. METHODS: An OSCE-type simulation assessment program was developed incorporating six stations representing basic and advanced surgical skills that are essential in trauma surgery. Each station was designed using ex-vivo animal tissue. The stations included basic knots and sutures, bowel resection and anastomosis, vascular end-to-end anastomosis, lung injury repair, cardiac injury repair, and laparoscopic suturing. Eight postgraduate year 2 (PY-2), eight recently graduated surgeons (RGS), and 3 experts were recruited, and their performance was blindly assessed by experts using the validated general rating scale OSATS (Objective Structured Assessment of Technical Skills) as well as the time taken to complete the procedure. RESULTS: Significant differences were identified among groups. The average OSATS score was 82 for the PY2 group, 113 for the RGS group, and 147 for the experts (P < 0.01). The average procedural time to complete all the stations was 98 minutes for the PY2 group, 68 minutes for the RGS group, and 35 minutes for the expert surgeons (P < 0.01). CONCLUSION: An OSCE scenario designed using ex-vivo tissue met 4 out of 5 criteria of the Messick validity framework: content, relation to other variables, response process and consequences of the test. The results show it is a valid strategy for the evaluation of practical skills in trauma surgery.
BACKGROUND: Trauma is one of the main causes of death globally, and appropriate surgical care is crucial to impact mortality. However, resident-performed trauma cases have diminished in the last 10 years. Simulation-based tools have proven to be effective to evaluate practical skills in a variety of settings. However, there is a lack of evidence regarding proper validation of trauma surgery models. OBJECTIVE: The aim of this study was to evaluate under a contemporary validity framework, an objective structured clinical evaluation (OSCE) scenario for the assessment of basic and advanced surgical skills in trauma and emergency surgery. METHODS: An OSCE-type simulation assessment program was developed incorporating six stations representing basic and advanced surgical skills that are essential in trauma surgery. Each station was designed using ex-vivo animal tissue. The stations included basic knots and sutures, bowel resection and anastomosis, vascular end-to-end anastomosis, lung injury repair, cardiac injury repair, and laparoscopic suturing. Eight postgraduate year 2 (PY-2), eight recently graduated surgeons (RGS), and 3 experts were recruited, and their performance was blindly assessed by experts using the validated general rating scale OSATS (Objective Structured Assessment of Technical Skills) as well as the time taken to complete the procedure. RESULTS: Significant differences were identified among groups. The average OSATS score was 82 for the PY2 group, 113 for the RGS group, and 147 for the experts (P < 0.01). The average procedural time to complete all the stations was 98 minutes for the PY2 group, 68 minutes for the RGS group, and 35 minutes for the expert surgeons (P < 0.01). CONCLUSION: An OSCE scenario designed using ex-vivo tissue met 4 out of 5 criteria of the Messick validity framework: content, relation to other variables, response process and consequences of the test. The results show it is a valid strategy for the evaluation of practical skills in trauma surgery.
Authors: Catalina Ortiz; Javier Vela; Caterina Contreras; Francisca Belmar; Ivan Paul; Analia Zinco; Juan Pablo Ramos; Pablo Ottolino; Pablo Achurra; Nicolas Jarufe; Adnan Alseidi; Julian Varas Journal: Surg Endosc Date: 2022-03-02 Impact factor: 4.584