BACKGROUND: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is a series of five tasks with an objective scoring system. The purpose of this study was to estimate the interrater and test-retest reliability of the MISTELS metrics and to assess their internal consistency. METHODS: To determine interrater reliability, two trained observers scored 10 subjects, either live or on tape. Test-retest reliability was assessed by having 12 subjects perform two tests, the second immediately following the first. Interrater and test-retest reliability were assessed using intraclass correlation coefficients. Internal consistency between tasks was estimated using Cronbach's alpha. RESULTS: The interrater and test-retest reliabilities for the total scores were both excellent at 0.998 [95% confidence interval (CI), 0.985-1.00] and 0.892 (95% CI, 0.665-0.968), respectively. Cronbach's alpha for the first assessment of the test-retest was 0.86. CONCLUSIONS: The MISTELS metrics have excellent reliability, which exceeds the threshold level of 0.8 required for high-stakes evaluations. These findings support the use of MISTELS for evaluation in many different settings, including residency training programs.
BACKGROUND: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is a series of five tasks with an objective scoring system. The purpose of this study was to estimate the interrater and test-retest reliability of the MISTELS metrics and to assess their internal consistency. METHODS: To determine interrater reliability, two trained observers scored 10 subjects, either live or on tape. Test-retest reliability was assessed by having 12 subjects perform two tests, the second immediately following the first. Interrater and test-retest reliability were assessed using intraclass correlation coefficients. Internal consistency between tasks was estimated using Cronbach's alpha. RESULTS: The interrater and test-retest reliabilities for the total scores were both excellent at 0.998 [95% confidence interval (CI), 0.985-1.00] and 0.892 (95% CI, 0.665-0.968), respectively. Cronbach's alpha for the first assessment of the test-retest was 0.86. CONCLUSIONS: The MISTELS metrics have excellent reliability, which exceeds the threshold level of 0.8 required for high-stakes evaluations. These findings support the use of MISTELS for evaluation in many different settings, including residency training programs.
Authors: Nabeel A Arain; Genevieve Dulan; Deborah C Hogg; Robert V Rege; Cathryn E Powers; Seifu T Tesfay; Linda S Hynan; Daniel J Scott Journal: Surg Endosc Date: 2012-04-27 Impact factor: 4.584
Authors: J B Pagador; J Uson; M A Sánchez; J L Moyano; J Moreno; P Bustos; J Mateos; F M Sánchez-Margallo Journal: Int J Comput Assist Radiol Surg Date: 2010-08-11 Impact factor: 2.924
Authors: Amine Chellali; Helena Mentis; Amie Miller; Woojin Ahn; Venkata S Arikatla; Ganesh Sankaranarayanan; Suvranu De; Steven D Schwaitzberg; Caroline G L Cao Journal: Int J Hum Comput Stud Date: 2016-07-09 Impact factor: 3.632
Authors: Daniel J Scott; E Matt Ritter; Seifu T Tesfay; Elisabeth A Pimentel; Alykhan Nagji; Gerald M Fried Journal: Surg Endosc Date: 2008-02-13 Impact factor: 4.584