Comparative research on training simulators in emergency medicine: a methodological review.

SUMMARY STATEMENT: Recent reviews and research agenda in simulation-based training for health care have repeatedly called for more studies comparing the effectiveness of different simulation modalities and approaches. Although a small body of such comparative research does exist, it is apparent that this line of research comes with particular methodological challenges. In this review, 20 studies comparing simulation modalities in emergency medicine were analyzed in terms of key methodological factors. Results of this literature review show that (1) past studies are largely underpowered to detect effects of the magnitude likely to be found, (2) researchers seeking to demonstrate equivalence of training approaches do not use appropriate statistical tests of equivalence, and (3) studies often use performance criterion test beds that may not support valid conclusions about trainees' future performance on the job. We discuss strengths and weaknesses of reviewed studies along these issues and recommend options for improving future comparative research in simulation-based training.