INTRODUCTION: Global Rating Scales (GRS) quantify and structure subjective expert assessment of skill. Hybrid simulators measure performance during physical laparoscopic tasks through instrument motion analysis. We assessed whether motion analysis metrics were as accurate as structured expert opinion by using GRS. METHODS: A random sample of 10 consultant laparoscopic surgeons, 10 senior trainees, and 10 novice students were assessed on a Sharp Dissection task. Coded video footage was reviewed by two blinded assessors and scored using a Likert Scale. Correlation with metrics was tested using Spearman's rho. Inter-rater reliability was measured using intraclass correlation coefficient (ICC). RESULTS: Strongest GRS-Metric correlations were found for Time/Motion/Progress with Time (Spearman's rho 0.88; p < 0.05) and Instrument Handling with Path Length (Spearman's rho 0.8; p < 0.05). Smoothness correlated with Respect for Tissue in Rater 1 (rho 0.68) but not Rater 2 (rho 0.18). Mean GRS showed stronger inter-rater agreement than individual scale components (ICC 0.68). Correlation coefficients with actual experience group were 0.58-0.74 for mean GRS score and 0.67-0.78 for metrics (Spearman's rho, p < 0.05). CONCLUSIONS: Metrics correlate well with GRS assessment, supporting concurrent validity. Metrics predict experience level as accurately as global rating and are construct valid. Hybrid simulators could provide resource-efficient feedback, freeing trainers to concentrate on teaching.
INTRODUCTION: Global Rating Scales (GRS) quantify and structure subjective expert assessment of skill. Hybrid simulators measure performance during physical laparoscopic tasks through instrument motion analysis. We assessed whether motion analysis metrics were as accurate as structured expert opinion by using GRS. METHODS: A random sample of 10 consultant laparoscopic surgeons, 10 senior trainees, and 10 novice students were assessed on a Sharp Dissection task. Coded video footage was reviewed by two blinded assessors and scored using a Likert Scale. Correlation with metrics was tested using Spearman's rho. Inter-rater reliability was measured using intraclass correlation coefficient (ICC). RESULTS: Strongest GRS-Metric correlations were found for Time/Motion/Progress with Time (Spearman's rho 0.88; p < 0.05) and Instrument Handling with Path Length (Spearman's rho 0.8; p < 0.05). Smoothness correlated with Respect for Tissue in Rater 1 (rho 0.68) but not Rater 2 (rho 0.18). Mean GRS showed stronger inter-rater agreement than individual scale components (ICC 0.68). Correlation coefficients with actual experience group were 0.58-0.74 for mean GRS score and 0.67-0.78 for metrics (Spearman's rho, p < 0.05). CONCLUSIONS: Metrics correlate well with GRS assessment, supporting concurrent validity. Metrics predict experience level as accurately as global rating and are construct valid. Hybrid simulators could provide resource-efficient feedback, freeing trainers to concentrate on teaching.
Authors: M C Vassiliou; G A Ghitulescu; L S Feldman; D Stanbridge; K Leffondré; H H Sigman; G M Fried Journal: Surg Endosc Date: 2006-02-27 Impact factor: 4.584
Authors: Anthony G Gallagher; C Daniel Smith; Steven P Bowers; Neal E Seymour; Adam Pearson; Steven McNatt; David Hananel; Richard M Satava Journal: J Am Coll Surg Date: 2003-09 Impact factor: 6.113
Authors: Maciej Kasprzyk; Michał Łuczak; Nel Kaczmarek; Jakub Psiuk; Marta Twardowska; Piotr Czarnecki Journal: Wideochir Inne Tech Maloinwazyjne Date: 2020-07-16 Impact factor: 1.195