OBJECTIVE: Pediatric residency programs need objective methods of trainee assessment. Patient simulation can contribute to objective evaluation of acute care event management skills. We describe the development and validation of 4 simulation case scenarios for pediatric resident evaluation. METHODS: We created 4 pediatric simulation cases: apnea, asthma, supraventricular tachycardia, and sepsis. Each case contains a scenario and an unweighted checklist. Case and checklist development began by reaching expert consensus about case content followed by 92 pilot simulation sessions used for content revision and rater training. After development, 54 first-and second-year pediatric residents participated in 108 simulation test cases to assess the validity of data from these tools for our population. We report outcomes for interrater reliability, discriminant validity, and the impact of potential confounding factors on validity estimates. RESULTS: Interrater reliability (kappa) ranged from 0.75 to 0.87. There were statistically and educationally significant differences in summary scores between first-and second-year residents for 3 of the 4 cases. Neither previous simulation exposure nor the order in which the cases were performed were found to be significant factors by multivariate analysis. CONCLUSIONS: Simulation can be used to reliably measure and discriminate resident competencies in acute care management. Rigorous measurement development work is difficult and time-consuming. Done correctly, measurement development yields tangible and lasting benefits for trainees, faculty, and residency programs. Development studies that use systematic procedures and large trainee samples at multiple sites are the best approach to creating measurement tools that yield valid data.
OBJECTIVE: Pediatric residency programs need objective methods of trainee assessment. Patient simulation can contribute to objective evaluation of acute care event management skills. We describe the development and validation of 4 simulation case scenarios for pediatric resident evaluation. METHODS: We created 4 pediatric simulation cases: apnea, asthma, supraventricular tachycardia, and sepsis. Each case contains a scenario and an unweighted checklist. Case and checklist development began by reaching expert consensus about case content followed by 92 pilot simulation sessions used for content revision and rater training. After development, 54 first-and second-year pediatric residents participated in 108 simulation test cases to assess the validity of data from these tools for our population. We report outcomes for interrater reliability, discriminant validity, and the impact of potential confounding factors on validity estimates. RESULTS: Interrater reliability (kappa) ranged from 0.75 to 0.87. There were statistically and educationally significant differences in summary scores between first-and second-year residents for 3 of the 4 cases. Neither previous simulation exposure nor the order in which the cases were performed were found to be significant factors by multivariate analysis. CONCLUSIONS: Simulation can be used to reliably measure and discriminate resident competencies in acute care management. Rigorous measurement development work is difficult and time-consuming. Done correctly, measurement development yields tangible and lasting benefits for trainees, faculty, and residency programs. Development studies that use systematic procedures and large trainee samples at multiple sites are the best approach to creating measurement tools that yield valid data.
Authors: J B Pagador; J Uson; M A Sánchez; J L Moyano; J Moreno; P Bustos; J Mateos; F M Sánchez-Margallo Journal: Int J Comput Assist Radiol Surg Date: 2010-08-11 Impact factor: 2.924
Authors: Joseph B House; Suzanne Dooley-Hash; Terry Kowalenko; Athina Sikavitsas; Desiree M Seeyave; John G Younger; Stanley J Hamstra; Michele M Nypaver Journal: J Grad Med Educ Date: 2012-09
Authors: David Drummond; Jennifer Truchot; Eleonora Fabbro; Pierre-François Ceccaldi; Patrick Plaisance; Antoine Tesnière; Alice Hadchouel Journal: Eur J Pediatr Date: 2017-12-04 Impact factor: 3.183
Authors: Cynthia M LaFond; Catherine Van Hulle Vincent; Sangyoon Lee; Colleen Corte; Patricia E Hershberger; Andrew Johnson; Chang G Park; Diana J Wilkie Journal: Simul Healthc Date: 2015-02 Impact factor: 1.929