Alexander Winkler-Schwartz1, Ibrahim Marwa2, Khalid Bajunaid3, Muhammad Mullah4, Fahad E Alotaibi5, Abdulgadir Bugdadi6, Robin Sawaya2, Abdulrahman J Sabbagh7, Rolando Del Maestro2. 1. Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada. Electronic address: manuscriptinquiry@gmail.com. 2. Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada. 3. Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Division of Neurosurgery, Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia. 4. Department of Epidemiology, Biostatistics & Occupational Health, McGill University, Montreal, Quebec, Canada. 5. Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Neurosurgical Department, National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia. 6. Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology & Neurosurgery, McGill University, Montreal, Quebec, Canada; Department of Surgery, Faculty of Medicine, Umm Al Qura University, Makkah, Saudi Arabia. 7. Division of Neurosurgery, Department of Surgery, Faculty of Medicine, University of Jeddah, Jeddah, Saudi Arabia; Clinical Skills and Simulation Center, King Abdulaziz University, Jeddah, Saudi Arabia.
Abstract
BACKGROUND: Adequate assessment and feedback remains a cornerstone of psychomotor skills acquisition, particularly within neurosurgery where the consequence of adverse operative events is significant. However, a critical appraisal of the reliability of visual rating scales in neurosurgery is lacking. Therefore, we sought to design a study to compare visual rating scales with simulated metrics in a neurosurgical virtual reality task. METHODS: Neurosurgical faculty rated anonymized participant video recordings of the removal of simulated brain tumors using a visual rating scale made up of seven composite elements. Scale reliability was evaluated using generalizability theory, and scale subcomponents were compared with simulated metrics using Pearson correlation analysis. RESULTS: Four staff neurosurgeons evaluated 16 medical student neurosurgery applicants. Overall scale reliability and internal consistency were 0.73 and 0.90, respectively. Reliability of 0.71 was achieved with two raters. Individual participants, raters, and scale items accounted for 27%, 11%, and 0.6% of the data variability. The hemostasis scale component related to the greatest number of simulated metrics, whereas respect for no-go zones and tissue was correlated with none. Metrics relating to instrument force and patient safety (brain volume removed and blood loss) were captured by the fewest number of rating scale components. CONCLUSIONS: To our knowledge, this is the first study comparing participant's ratings with simulated performance. Given rating scales capture less well instrument force, quantity of brain volume removed, and blood loss, we suggest adopting a hybrid educational approach using visual rating scales in an operative environment, supplemented by simulated sessions to uncover potentially problematic surgical technique.
BACKGROUND: Adequate assessment and feedback remains a cornerstone of psychomotor skills acquisition, particularly within neurosurgery where the consequence of adverse operative events is significant. However, a critical appraisal of the reliability of visual rating scales in neurosurgery is lacking. Therefore, we sought to design a study to compare visual rating scales with simulated metrics in a neurosurgical virtual reality task. METHODS: Neurosurgical faculty rated anonymized participant video recordings of the removal of simulated brain tumors using a visual rating scale made up of seven composite elements. Scale reliability was evaluated using generalizability theory, and scale subcomponents were compared with simulated metrics using Pearson correlation analysis. RESULTS: Four staff neurosurgeons evaluated 16 medical student neurosurgery applicants. Overall scale reliability and internal consistency were 0.73 and 0.90, respectively. Reliability of 0.71 was achieved with two raters. Individual participants, raters, and scale items accounted for 27%, 11%, and 0.6% of the data variability. The hemostasis scale component related to the greatest number of simulated metrics, whereas respect for no-go zones and tissue was correlated with none. Metrics relating to instrument force and patient safety (brain volume removed and blood loss) were captured by the fewest number of rating scale components. CONCLUSIONS: To our knowledge, this is the first study comparing participant's ratings with simulated performance. Given rating scales capture less well instrument force, quantity of brain volume removed, and blood loss, we suggest adopting a hybrid educational approach using visual rating scales in an operative environment, supplemented by simulated sessions to uncover potentially problematic surgical technique.
Authors: Ali M Fazlollahi; Mohamad Bakhaidar; Ahmad Alsayegh; Recai Yilmaz; Alexander Winkler-Schwartz; Nykan Mirchi; Ian Langleben; Nicole Ledwos; Abdulrahman J Sabbagh; Khalid Bajunaid; Jason M Harley; Rolando F Del Maestro Journal: JAMA Netw Open Date: 2022-02-01
Authors: Alessandro Iop; Victor Gabriel El-Hajj; Maria Gharios; Andrea de Giorgio; Fabio Marco Monetti; Erik Edström; Adrian Elmi-Terander; Mario Romero Journal: Sensors (Basel) Date: 2022-08-14 Impact factor: 3.847
Authors: Maryam Pishgar; Salah Fuad Issa; Margaret Sietsema; Preethi Pratap; Houshang Darabi Journal: Int J Environ Res Public Health Date: 2021-06-22 Impact factor: 3.390