BACKGROUND: Simulator performance is measured by metrics, which are valued as an objective way of assessing trainees. Certain procedures such as laparoscopic suturing, however, may not be suitable for assessment under traditionally formulated metrics. Our aim was to assess if our new metric is a valid method of assessing laparoscopic suturing. STUDY DESIGN: A software program was developed to order to create a new metric, which would calculate the percentage of time spent operating within pre-defined areas called "zones." Twenty-five candidates (medical students N = 10, surgical residents N = 10, and laparoscopic experts N = 5) performed the laparoscopic suturing task on the ProMIS III(®) simulator. New metrics of "in-zone" and "out-zone" scores as well as traditional metrics of time, path length, and smoothness were generated. Performance was also assessed by two blinded observers using the OSATS and FLS rating scales. This novel metric was evaluated by comparing it to both traditional metrics and subjective scores. RESULTS: There was a significant difference in the average in-zone and out-zone scores between all three experience groups (p < 0.05). The new zone metrics scores correlated significantly with the subjective-blinded observer scores of OSATS and FLS (p = 0.0001). The new zone metric scores also correlated significantly with the traditional metrics of path length, time, and smoothness (p < 0.05). CONCLUSION: The new metric is a valid tool for assessing laparoscopic suturing objectively. This could be incorporated into a competency-based curriculum to monitor resident progression in the simulated setting.
BACKGROUND: Simulator performance is measured by metrics, which are valued as an objective way of assessing trainees. Certain procedures such as laparoscopic suturing, however, may not be suitable for assessment under traditionally formulated metrics. Our aim was to assess if our new metric is a valid method of assessing laparoscopic suturing. STUDY DESIGN: A software program was developed to order to create a new metric, which would calculate the percentage of time spent operating within pre-defined areas called "zones." Twenty-five candidates (medical students N = 10, surgical residents N = 10, and laparoscopic experts N = 5) performed the laparoscopic suturing task on the ProMIS III(®) simulator. New metrics of "in-zone" and "out-zone" scores as well as traditional metrics of time, path length, and smoothness were generated. Performance was also assessed by two blinded observers using the OSATS and FLS rating scales. This novel metric was evaluated by comparing it to both traditional metrics and subjective scores. RESULTS: There was a significant difference in the average in-zone and out-zone scores between all three experience groups (p < 0.05). The new zone metrics scores correlated significantly with the subjective-blinded observer scores of OSATS and FLS (p = 0.0001). The new zone metric scores also correlated significantly with the traditional metrics of path length, time, and smoothness (p < 0.05). CONCLUSION: The new metric is a valid tool for assessing laparoscopic suturing objectively. This could be incorporated into a competency-based curriculum to monitor resident progression in the simulated setting.
Authors: Dimitrios Stefanidis; William W Hope; James R Korndorffer; Sarah Markley; Daniel J Scott Journal: J Am Coll Surg Date: 2010-04 Impact factor: 6.113
Authors: Christopher W Snyder; Marianne J Vandromme; Sharon L Tyra; John R Porterfield; Ronald H Clements; Mary T Hawn Journal: World J Surg Date: 2011-02 Impact factor: 3.352
Authors: Paul C Neary; Emily Boyle; Conor P Delaney; Anthony J Senagore; Frank B V Keane; Anthony G Gallagher Journal: Surg Endosc Date: 2008-06-14 Impact factor: 4.584
Authors: Karl-Friedrich Kowalewski; Andreas Minassian; Jonathan David Hendrie; Laura Benner; Anas Amin Preukschas; Hannes Götz Kenngott; Lars Fischer; Beat P Müller-Stich; Felix Nickel Journal: Surg Endosc Date: 2018-09-07 Impact factor: 4.584