Justues Chang1,2, Daniel C Banaszek3,4, Jason Gambrel5, Davide Bardana6,7. 1. School of Medicine, Queen's University, Kingston, ON, Canada. 2. Division of Orthopaedic Surgery, University of Toronto, Toronto, ON, Canada. 3. School of Medicine, Queen's University, Kingston, ON, Canada. 12db24@queensu.ca. 4. Division of Orthopaedic Surgery, Queen's University, 905 Roshan Drive, Kingston, ON, K7P 0B1, Canada. 12db24@queensu.ca. 5. Brockville General Hospital, Brockville, ON, Canada. 6. Surgical Foundations Program, Division of Orthopaedic Surgery, Queen's University, Kingston, ON, Canada. 7. Kingston General Hospital, Kingston, ON, Canada.
Abstract
BACKGROUND: Work-hour restrictions and fatigue management strategies in surgical training programs continue to evolve in an effort to improve the learning environment and promote safer patient care. In response, training programs must reevaluate how various teaching modalities such as simulation can augment the development of surgical competence in trainees. For surgical simulators to be most useful, it is important to determine whether surgical proficiency can be reliably differentiated using them. To our knowledge, performance on both virtual and benchtop arthroscopy simulators has not been concurrently assessed in the same subjects. QUESTIONS/PURPOSES: (1) Do global rating scales and procedure time differentiate arthroscopic expertise in virtual and benchtop knee models? (2) Can commercially available built-in motion analysis metrics differentiate arthroscopic expertise? (3) How well are performance measures on virtual and benchtop simulators correlated? (4) Are these metrics sensitive enough to differentiate by year of training? METHODS: A cross-sectional study of 19 subjects (four medical students, 12 residents, and three staff) were recruited and divided into 11 novice arthroscopists (student to Postgraduate Year [PGY] 3) and eight proficient arthroscopists (PGY 4 to staff) who completed a diagnostic arthroscopy and loose-body retrieval in both virtual and benchtop knee models. Global rating scales (GRS), procedure times, and motion analysis metrics were used to evaluate performance. RESULTS: The proficient group scored higher on virtual (14 ± 6 [95% confidence interval {CI}, 10-18] versus 36 ± 5 [95% CI, 32-40], p < 0.001) and benchtop (16 ± 8 [95% CI, 11-21] versus 36 ± 5 [95% CI, 31-40], p < 0.001) GRS scales. The proficient subjects completed nearly all tasks faster than novice subjects, including the virtual scope (579 ±169 [95% CI, 466-692] versus 358 ± 178 [95% CI, 210-507] seconds, p = 0.02) and benchtop knee scope + probe (480 ± 160 [95% CI, 373-588] versus 277 ± 64 [95% CI, 224-330] seconds, p = 0.002). The built-in motion analysis metrics also distinguished novices from proficient arthroscopists using the self-generated virtual loose body retrieval task scores (4 ± 1 [95% CI, 3-5] versus 6 ± 1 [95% CI, 5-7], p = 0.001). GRS scores between virtual and benchtop models were very strongly correlated (ρ = 0.93, p < 0.001). There was strong correlation between year of training and virtual GRS (ρ = 0.8, p < 0.001) and benchtop GRS (ρ = 0.87, p < 0.001) scores. CONCLUSIONS: To our knowledge, this is the first study to evaluate performance on both virtual and benchtop knee simulators. We have shown that subjective GRS scores and objective motion analysis metrics and procedure time are valid measures to distinguish arthroscopic skill on both virtual and benchtop modalities. Performance on both modalities is well correlated. We believe that training on artificial models allows acquisition of skills in a safe environment. Future work should compare different modalities in the efficiency of skill acquisition, retention, and transferability to the operating room.
BACKGROUND: Work-hour restrictions and fatigue management strategies in surgical training programs continue to evolve in an effort to improve the learning environment and promote safer patient care. In response, training programs must reevaluate how various teaching modalities such as simulation can augment the development of surgical competence in trainees. For surgical simulators to be most useful, it is important to determine whether surgical proficiency can be reliably differentiated using them. To our knowledge, performance on both virtual and benchtop arthroscopy simulators has not been concurrently assessed in the same subjects. QUESTIONS/PURPOSES: (1) Do global rating scales and procedure time differentiate arthroscopic expertise in virtual and benchtop knee models? (2) Can commercially available built-in motion analysis metrics differentiate arthroscopic expertise? (3) How well are performance measures on virtual and benchtop simulators correlated? (4) Are these metrics sensitive enough to differentiate by year of training? METHODS: A cross-sectional study of 19 subjects (four medical students, 12 residents, and three staff) were recruited and divided into 11 novice arthroscopists (student to Postgraduate Year [PGY] 3) and eight proficient arthroscopists (PGY 4 to staff) who completed a diagnostic arthroscopy and loose-body retrieval in both virtual and benchtop knee models. Global rating scales (GRS), procedure times, and motion analysis metrics were used to evaluate performance. RESULTS: The proficient group scored higher on virtual (14 ± 6 [95% confidence interval {CI}, 10-18] versus 36 ± 5 [95% CI, 32-40], p < 0.001) and benchtop (16 ± 8 [95% CI, 11-21] versus 36 ± 5 [95% CI, 31-40], p < 0.001) GRS scales. The proficient subjects completed nearly all tasks faster than novice subjects, including the virtual scope (579 ±169 [95% CI, 466-692] versus 358 ± 178 [95% CI, 210-507] seconds, p = 0.02) and benchtop knee scope + probe (480 ± 160 [95% CI, 373-588] versus 277 ± 64 [95% CI, 224-330] seconds, p = 0.002). The built-in motion analysis metrics also distinguished novices from proficient arthroscopists using the self-generated virtual loose body retrieval task scores (4 ± 1 [95% CI, 3-5] versus 6 ± 1 [95% CI, 5-7], p = 0.001). GRS scores between virtual and benchtop models were very strongly correlated (ρ = 0.93, p < 0.001). There was strong correlation between year of training and virtual GRS (ρ = 0.8, p < 0.001) and benchtop GRS (ρ = 0.87, p < 0.001) scores. CONCLUSIONS: To our knowledge, this is the first study to evaluate performance on both virtual and benchtop knee simulators. We have shown that subjective GRS scores and objective motion analysis metrics and procedure time are valid measures to distinguish arthroscopic skill on both virtual and benchtop modalities. Performance on both modalities is well correlated. We believe that training on artificial models allows acquisition of skills in a safe environment. Future work should compare different modalities in the efficiency of skill acquisition, retention, and transferability to the operating room.
Authors: James R Korndorffer; Daniel J Scott; Rafael Sierra; William C Brunner; J Bruce Dunne; Douglas P Slakey; Michael C Townsend; Robert L Hewitt Journal: Arch Surg Date: 2005-01
Authors: William C Brunner; James R Korndorffer; Rafael Sierra; J Bruce Dunne; C Lillian Yau; Ralph L Corsetti; Douglas P Slakey; Michael C Townsend; Daniel J Scott Journal: Am Surg Date: 2005-01 Impact factor: 0.688
Authors: Rajesh Aggarwal; Teodor P Grantcharov; Jens R Eriksen; Dorthe Blirup; Viggo B Kristiansen; Peter Funch-Jensen; Ara Darzi Journal: Ann Surg Date: 2006-08 Impact factor: 12.969
Authors: Gunnar Ahlberg; Lars Enochsson; Anthony G Gallagher; Leif Hedman; Christian Hogman; David A McClusky; Stig Ramel; C Daniel Smith; Dag Arvidsson Journal: Am J Surg Date: 2007-06 Impact factor: 2.565
Authors: Ryan J Koehler; Simon Amsdell; Elizabeth A Arendt; Leslie J Bisson; Jonathan P Braman; Jonathan P Bramen; Aaron Butler; Andrew J Cosgarea; Christopher D Harner; William E Garrett; Tyson Olson; Winston J Warme; Gregg T Nicandri Journal: Am J Sports Med Date: 2013-04-02 Impact factor: 6.202
Authors: Courtney D Bell; Joseph G O'Sullivan; Tamara E Ostervoss; William E Cameron; Ryan C Petering; Jacqueline M Brady Journal: J Grad Med Educ Date: 2020-06