BACKGROUND: There is increasing emphasis on performance-based assessment of clinical competence. The High Fidelity Patient Simulator (HPS) may be useful for assessment of clinical practice in anaesthesia, but needs formal evaluation of validity, reliability, feasibility and effect on learning. We set out to assess the reliability of a global rating scale for scoring simulator performance in crisis management. METHODS: Using a global rating scale, three judges independently rated videotapes of anaesthetists in simulated crises in the operating theatre. Five anaesthetists then independently rated subsets of these videotapes. RESULTS: There was good agreement between raters for medical management, behavioural attributes and overall performance. Agreement was high for both the initial judges and the five additional raters. CONCLUSIONS: Using a global scale to assess simulator performance, we found good inter-rater reliability for scoring performance in a crisis. We estimate that two judges should provide a reliable assessment. High fidelity simulation should be studied further for assessing clinical performance.
BACKGROUND: There is increasing emphasis on performance-based assessment of clinical competence. The High Fidelity Patient Simulator (HPS) may be useful for assessment of clinical practice in anaesthesia, but needs formal evaluation of validity, reliability, feasibility and effect on learning. We set out to assess the reliability of a global rating scale for scoring simulator performance in crisis management. METHODS: Using a global rating scale, three judges independently rated videotapes of anaesthetists in simulated crises in the operating theatre. Five anaesthetists then independently rated subsets of these videotapes. RESULTS: There was good agreement between raters for medical management, behavioural attributes and overall performance. Agreement was high for both the initial judges and the five additional raters. CONCLUSIONS: Using a global scale to assess simulator performance, we found good inter-rater reliability for scoring performance in a crisis. We estimate that two judges should provide a reliable assessment. High fidelity simulation should be studied further for assessing clinical performance.
Authors: Michael J Musacchio; Adam P Smith; Christopher A McNeal; Lorenzo Munoz; David M Rothenberg; Kelvin A von Roenn; Richard W Byrne Journal: Neurocrit Care Date: 2010-10 Impact factor: 3.210
Authors: Beverly W Henry; Danielle M McCarthy; Anna P Nannicelli; Nicholas P Seivert; John A Vozenilek Journal: Health Expect Date: 2013-10-07 Impact factor: 3.377
Authors: Brenda S Bray; Catrina R Schwartz; Peggy Soule Odegard; Dana P Hammer; Amy L Seybert Journal: Am J Pharm Educ Date: 2011-12-15 Impact factor: 2.047
Authors: Alice A Edler; Ruth G Fanning; Michael I Chen; Rebecca Claure; Dondee Almazan; Brain Struyk; Samuel C Seiden Journal: J Educ Eval Health Prof Date: 2009-12-20