Five Questions Critical Care Educators Should Ask About Simulation-Based Medical Education.

Simulation is now commonly used in health care education, and a growing body of evidence supports its positive impact on learning. However, simulation-based medical education (SBME) involves a range of modalities, instructional methods, and presentations associated with different advantages and limitations. This review aims at better understanding the nature of SBME, its theoretic and proven benefits, its delivery, and the challenges posed by SBME. Areas requiring further research and development are also discussed.

Key points

Simulation-based medical education (SBME) is an instructional medium that refers to the use of multiple simulation modalities.

SBME presents favorable characteristics for the achievement of educational goals that may not be fully addressed during clinical-based training.

Numerous studies support the use of SBME for the improvement of knowledge, technical and nontechnical tasks (teamwork, communication skills), and system issues in different clinical domains.

More research is needed to better understand the most effective use of SBME as part of a broader medical curriculum.

Introduction

Simulation is rapidly permeating into every sphere of medical education, including teaching, assessment, and research. Publications about SBME have grown exponentially over the past 10 years.1, 2 In addition, the quality of the studies published on SBME has consistently increased, as illustrated by a series of reviews published by Issenberg, McGaghie and colleagues.3, 4, 5 This impressive body of evidence now includes original studies, narrative and systematic reviews, opinion and position papers in different clinical domains, as well as a fewer number of publications specifically related to critical care medicine. These studies combined with the research findings in other clinical areas demonstrate the relevance of SBME for critical care health professionals.6, 7, 8, 9, 10, 11 In the presence of such an extensive literature, it seemed superfluous to undertake another systematic review on the topic of simulation in critical care medicine. However, this abundance of information on SBME can be overwhelming for critical care educators who attempt, in the midst of many other professional responsibilities, to design, implement, and evaluate sound educational innovations or curricula for their trainees. The focus of this review on SBME is therefore to summarize the evidence relevant for frontline educators in critical care medicine. The authors briefly examine 5 practical questions aimed at better understanding the nature of SBME, its theoretic and proven benefits, its delivery, as well as the challenges posed by SBME. The term SBME is used broadly to include education of all health care professionals. Although SBME has traditionally been predominantly focused on physicians, simulation studies are increasingly directed toward other health care professionals, such as nurses, pharmacists, and dieticians.12, 13, 14

What is simulation-based medical education?

Simulation has been broadly defined as “an instructional medium used for education, assessment, and research, which includes several modalities that have in common the reproduction of certain characteristics of the clinical reality.” Simulation modalities typically include part-task trainers (interactions with a physical or virtual model requiring the use of specific psychomotor skills to complete procedures), human simulation (interactions with a simulated or standardized patient), computer-based simulation (interaction with a screen-based interface), and simulated clinical immersion (interaction with a physical or virtual work environment including team members, computer-driven manikin patient, and equipment).9, 10, 15, 16, 17 More recently, simulation modalities have been combined into hybrid simulations to facilitate the simultaneous and integrative practice of complementary skills (eg, part-task trainer on a simulated patient to practice suturing and communication skills). In addition, simulators have been placed in real clinical environments to create in situ simulations. In the face of such variety of simulation modalities, medical educators need to choose a modality that best aligns with the learning objectives of their training programs and depending upon the benefits and limitations of each modality.9, 15

Why should simulation-based medical education be used?

If simulation has ever elicited doubts among health care educators about its roles in medical education, many think that we have now moved beyond the need to justify its use.13, 20, 21 Based on a growing body of evidence, the use of simulation seems judicious for certain aspects of health care training. The need to better define how to use SBME optimally and cost effectively has been identified as the next question to be answered by the medical education community.7, 13, 20

It is worthwhile to briefly consider the context in which SBME rapidly gained popularity over the recent years, after initial delays in the uptake of a technology developed more than three decades ago.7, 9 As medical education evolved, educators developed a better understanding of the learning processes involved in health care education, such as deliberate practice,22, 23 reflection,24, 25, 26 and feedback.27, 28, 29 Meanwhile, the traditional apprenticeship model, based on the prolonged and repeated interactions between junior and senior health care professionals, was increasingly under threat because of numerous changes in the health care system: increased clinical workload; rapid turnover of patients and health care professionals in a given clinical unit; competing academic roles of faculty as clinicians, educators, and researchers; and so on.9, 30, 31, 32 Such changes led to perceived inadequacies of the medical training system: the key processes thought to benefit clinical learning seemed increasingly hard to experience in acute care environments.7, 32 Furthermore, growing concerns about patient safety made the idea of inexperienced trainees practicing their skills on real patients morally inacceptable.17, 31, 33 Working hour limitations and increased patient supervision were implemented to increase the quality of care provided in teaching hospitals.9, 17, 32, 33, 34, 35 The educational shortcomings of this new form of time-limited, autonomy-restricted clinical experience needed to be compensated.31, 36 In addition to individual competencies, human factors and teamwork were identified as a common source of medical errors.9, 37 Specific training for interdisciplinary teams was strongly recommended to further improve the quality of care.

Simulation represented a natural response to these problems for several reasons.17, 38 First, simulation can provide a safe learning environment where mistakes can be made, reviewed, corrected, and reflected upon.9, 17 Second, simulation offers the opportunity to practice clinical skills and to achieve a certain level of proficiency or mastery before caring of real patients. Third, simulation offers greater control and predictability of the learning experience in terms of type, order, number, and length of the sessions; type of feedback provided; number and level of participants; and so on. In theory, SBME therefore presents desirable characteristics to meet the educational needs of health care professionals in training, as well as the moral obligation to prioritize patient safety in real clinical situations.9, 17

The medical education literature provides clear evidence that SBME can fulfill an important role within health care professional training. At least 3 systematic reviews have now demonstrated that SBME, as an instructional medium, can positively affect learning and can translate into benefits for the patients.13, 14, 39 However, the heterogeneity of the literature on SBME justifies taking a closer look at the potential educational benefits of SBME. This topic is the focus of the following section.

What can be taught and assessed with simulation-based medical education?

Simulation-based education has been used to teach and study a broad range of knowledge, skills, and attitudes. Multiple reviews have attempted to summarize the current evidence supporting the use of SBME. Some of these reviews are specific for critical care medicine,6, 9, 10, 11, 40 whereas others are related to different clinical specialties, such as anesthesia,1, 41 obstetrics, emergency medicine,17, 43 surgery, or pediatrics. Specific modalities or groups of modalities have also been the object of literature reviews, including simulated patients, virtual simulation, technology-enhanced simulation, computer-assisted learning, and in situ simulation. The following conclusions can be derived from the examination of these reviews:

Simulation is generally well accepted by learners as a teaching strategy, as illustrated by the positive ratings consistently reported in trainee satisfaction surveys.

The types of learning outcomes measured to demonstrate the benefits of SBME are not equally represented in the literature: studies showing short-term gain in knowledge and skills in the simulation environment (levels 1 and 2 of Kirkpatrick's model presented in Box 1 ) are overrepresented when compared with studies assessing long-term (eg, 6–12 months) gain of knowledge and skills, or changes in behaviors transferred to real clinical environments and benefiting patients (levels 3 and 4 of Kirkpatrick's model).

Level 1: Reaction

Level of satisfaction regarding training

Level 2: Learning

Knowledge, skills, and attitudes acquired

Level 3: Behavior

Transfer of learning to workplace

Level 4: Results

Transfer or impact on society (patients)

The quality of the evidence on SBME is generally limited, as indicated by small sample sizes, lack of control group or randomization, scarcity of multicenter studies, and poor reporting.13, 51

Improvement of knowledge, technical and nontechnical tasks, teamwork, communication skills, and system issues has been achieved by SBME in certain areas.

SBME has shown promising results in many areas relevant for critical care educators. Specific examples related to technical and nontechnical skills, system issues, and assessment tools are presented in Box 2 . When appropriate, the Kirkpatrick's level of learning outcomes measured by individual studies is indicated. Simulation is also being increasingly considered in 2 other educational domains: high-stakes assessment and mass casualty training. Simulation-based assessment is not a new phenomenon and has been incorporated in licensure and certification examination at the undergraduate and postgraduate level for many years. More recently, simulation has also been used in regulatory programs for practicing physicians in the fields of anesthesia, internal medicine, and family medicine. In addition to part-task trainers and simulated patients, full-body manikin simulators are now increasingly used for high-stakes assessment. As the caveats of simulation-based assessment are better understood and slowly overcome (eg, psychometric properties and technological limitations), simulation will likely play an important role in high-stakes assessment in critical care. Finally, SBME has been shown to be useful for mass casualty training and may represent a useful tool for health care professional training in response to specific threats, such as the recent Ebola epidemic.

Procedures

Central line insertion (KL 1–4)46, 64, 65, 66, 67

Airway management and endotracheal intubation (KL 2, 3, 4)68, 69, 70, 71

Bronchoscopy (KL 2–3)72, 73

Cricothyroidotomy (KL 2)

Thoracocentesis (KL 2)

Paracentesis (KL 2)

Ultrasonography skills, including echocardiography (KL 1, 2)77, 78

Task-related technical skills

Neurocritical skills (KL 1)

ACLS skills (KL 1–4)14, 80, 81, 82, 83

Task management of critical care crises (KL 1, 2)84, 85, 86, 87, 88, 89

Task management of patients with trauma (KL 1, 2)84, 90

Knowledge and skills in respiratory mechanics, mechanical ventilation, and circulation (KL 3)

Crisis resource management skills (KL 1, 2)40, 92, 93, 94, 95

Team behaviors during ATLS (KL 2)90, 96

Team crisis responses, including outcomes of simulated patient and communication (KL 2)

Interprofessional team responses of undergraduate students (KL 1)

MET performance (KL 2, 4)

Teamwork during postcardiac surgery pediatric cardiac arrest (KL 1)

Interphysician variability in ICU admission of patients with end-stage cancer

Nursing hand-offs of patients in ICU

Discrepancies between institutional or departmental policies and clinical practice related to obstetric emergencies

Infectious disease challenges during SARS cardiac arrests

Teamwork

Mayo High Performance Teamwork scale for CRM skills

Self-assessment tool of teamwork in critical care

Clinical Teamwork Scale

Crisis management behavior performance markers

TEAM

Individual nontechnical skills

Anesthesia nontechnical skills system

Ottawa Global Rating Scale for nontechnical skills111, 112

Family Conference OSCE for professionalism and communication skills

Checklist for professionalism during ethical dilemma

Individual technical or mixed skills

Interdisciplinary management of septic shock

Integrated Procedural Performance Instrument for technical and communication skills

Scenario-specific performance checklist for pediatric scenarios, undergraduates, and acute care scenarios119, 120

IPETT for emergency technical and nontechnical skills

Comparison between written examination, simulation-based, and oral viva examinations for procedural skills

Abbreviations: ACLS, Advanced Cardiac Life Support; ATLS, Advanced Trauma Life Support; CRM, crisis resource management; ICU, intensive care unit; IPETT, Imperial Pediatric Emergency Training Toolkit; KL, Kirkpatrick's level; MET, medical emergency team; OSCE, Objective Structured Clinical Examination; SARS, severe acute respiratory syndrome; TEAM, Team Emergency Assessment Measure.

There are also studies in the domains of neonatal resuscitation,56, 57 teamwork during cardiac arrests, airway management, and Advanced Trauma Life Support (ATLS) skills that have failed to demonstrate a clear benefit of SBME. Most of these studies have compared SBME with other types of educational interventions such as video training, case-based or problem-based discussions, and traditional teaching. Limitations in the methodological rigor of many studies supporting the role of SBME, as well as inconsistencies regarding its efficacy when compared with other learning strategies, call for thoughtful reflection as to when simulation should, or should not, be used for critical care instruction. In this regard, another type of literature on SBME, described in the following section, helps to further inform the judicious use of SBME in critical care medicine.

How should simulation-based medical education be used?

SBME is a time-consuming, potentially costly enterprise and must therefore be carefully planned to maximize its educational benefits and minimize its resource requirements. Thankfully, the body of literature dedicated to the understanding of the features characterizing effective SBME is slowly growing. Box 3 summarizes the types of questions that critical care educators should consider when planning a simulation-based curricular activity.4, 5

How is this intervention integrated with other aspects of the curriculum?

Based on the learning objectives, should this training be interdisciplinary and/or focused on team training?

How will the facilitator/instructor be chosen and trained (clinical, educational, and interpersonal skills)?

How should the right level of simulation fidelity be chosen (physical, psychological, and sociocultural)?

How much and what type of practice will be required of the participants (repetitive, deliberate, massed, or distributed)?

How will the feedback be provided (by whom, when, how often, how)?

How will each practice session differ from the other (progressive difficulty, variety of cases, adapted to individual learners)?

When will the intervention end (achievement of mastery learning)?

How and where will the outcomes of this intervention be measured (in the simulation setting or in real clinical environments)?

How can it be ensured that the knowledge and skills acquired will be maintained?

Chiniara and colleagues have also nicely summarized many issues related to the instructional design of SBME. The investigators discussed the learning objectives for which simulation may be the most appropriate medium, the choice of a simulation modality, the choice of an instructional method (self-directed or instructor-based method), and the simulation presentation (including feedback, fidelity, type of simulator, scenarios, and team composition).

Studies assessing the best ways to deliver SBME have been increasingly conducted in the critical care setting. For example, Springer and colleagues concluded that multiple 30-min simulation sessions held over 3 consecutive days were more effective than one 90-min session to improve resident knowledge regarding recognition and management of septic shock. Ali and colleagues reported that both students and instructors perceived the use of mechanical simulators and of simulated patients as equally satisfactory for ATLS training. Such studies can contribute to the improvement of SBME by increasing its efficacy or reducing its costs. However, there are still a large number of unanswered questions regarding the best ways to optimize the use of SBME. This is an area that requires further high-quality research.

What are the challenges related to simulation-based medical education?

As described earlier, the rising popularity of SBME has emerged in a specific context where traditional clinical-based training is increasingly challenging because of complex and interdependent societal and organizational changes. Based on the current evidence, the use of SBME to prevent medical errors and adverse events seems totally legitimate. However, the authors foresee important challenges that critical care educators should carefully consider before engaging in SBME activities.

First, the authors feel the need to address the aspects of health care professional training that SBME will not address. Clinical training is challenging in part because of the lack of availability of dedicated clinical teachers (who struggle to fulfill other professional responsibilities) and motivated trainees (overwhelming workload, working hours limitation, etc.). Health care resources are globally limited. The lack of time and scarcity of human and financial resources identified as barriers for clinical learning also apply to SBME. Clinically competent, properly trained simulation instructors are a rarity in most institutions. Training faculty to fulfill these responsibilities is time and resource consuming. Furthermore, the authors do not share the enthusiasm of others regarding the potential to add hours of SBME to the working hours of medical trainees. The educational value of any training completed beyond 80 hours of clinical work is questionable. Although the mistakes committed in a simulated environment will not harm any patient, tired trainees are unlikely to efficiently learn and to positively process their simulation experience. Furthermore, the time and energy invested in SBME necessarily redirect part of the energy and resources from clinical-based training toward SBME. Such unilateral shift could potentially represent additional impediments to the improvement of clinical-based learning, still recognized, even among the strongest proponents of SBME, as a core component of health care education. The authors believe that clinical-based learning and SBME can inform and complement each other in many aspects of training. In their opinion, recent calls for better curricular integration of SBME are an encouraging step in the right direction. SBME is not a panacea that will fix all the medical education problems, and clinical-based training should continue to be a high priority for medical educators and researchers.

The second challenge faced by SBME is the significant gap between the theoretic understanding of how SBME should be delivered and the way it is currently delivered in most institutions. With the exception of a few programs led by groups of committed and trained educators, SBME is frequently delivered in an ad hoc and unsystematic manner, separate from the broader curriculum. Box 4 suggests an approach to SBME based on general principles of curriculum development. Far from wanting to blame educators, their intention is rather to highlight how difficult it can be for an educational intervention to present most of the features of effective SBME. In an attempt to be pragmatic about the implementation of SBME, educators must often select one of 2 features on which to focus their time and energy. The real benefits of SBME as currently applied may therefore be less than the ones presented in the literature.

Identification of an educational problem

Which educational need is not adequately addressed by the current curriculum?

How is this problem currently addressed?

Could SBME help address this problem?

Targeted need assessment

Which learners should be targeted for this program/activity?

What are the learners' specific needs?

Is a simulated learning environment appropriate to address these needs?

Goals and objectives

What are the general goals of this program/activity?

What are the measurable objectives that will be achieved?

Educational strategies

Which simulation modality is the most appropriate to achieve these goals?

Which instructional method will be the most helpful?

How should the simulation activity/program be presented (timing, duration, feedback, etc.)?

Implementation

Which kind of support/resources will be required?

Which barriers to implementation can be expected?

Evaluation and feedback

How will feedback be obtained from individual learners?

How will this activity be assessed at the program level?

Summary

SBME has come a long way since the introduction of the first simulator more than 30 years ago. The authors have many reasons to be optimistic about SBME: the role of SBME is expanding (identification of gaps in clinical training and practice; undergraduate, postgraduate, and continuing education; formative and high-stakes assessment), the quality of evidence to support its use is increasing, and the strategies to implement SBME effectively and efficiently are better understood. There seems to be a consensus in the literature that SBME has an important role to play in the improvement of the safety of the care delivered in health care institutions; patient outcomes can be affected by our educational choices. These conclusions likely apply to the critical care environment in which patient outcomes critically depend on timely, complex, and highly coordinated care. However, significant research is still needed to further explore the advantages and limitations of SBME for specific clinical activities completed in particular clinical contexts. Such efforts should be coordinated with larger educational initiatives aimed at providing the best and most comprehensive educational experience for the critical care trainees.