Mastery in Simulation in Critical Care before Transitioning to Practice. Are There Drawbacks?

William Osler once stated: “To study the phenomenon of disease without books is to sail an uncharted sea, while to study books without patients is not to go to sea at all” (1). Osler was one in a long line of master clinicians who emphasized the importance of integrating theoretical learning with bedside experience.

In addition to books, innovations in educational technology and instruction offer new learning opportunities, such as computer-based learning, virtual patients, and simulations. In parallel with these changes, medical education is evolving from a time-based model to a competency-based model. Competency-based medical education (CBME) relies on outcome measurements to guide high-stakes decisions. The traditional bedside method of skill training known as “see one, do one, teach one” has been criticized heavily, and the absence of simulation training is now deemed unethical, in view of concerns for patient safety and advances in technology affording sophisticated simulation curricula (2). In the era of CBME, mastery learning has gained popularity. Simulation-based mastery learning (SBML) requires that trainees reach a predetermined standard of performance in simulation before progressing to supervised bedside practice (3).

Considering the strong endorsement that SBML is receiving in emerging clinical education models in CBME, our commentary is directed at examining the following:

The feasibility of this approach for a critical-care-medicine (CCM) training program, by examining the time that a critical-care trainee would require in simulation for skills training before transitioning to supervised bedside practice.

Whether the SBML model, which requires trainees to reach a predetermined level of performance in simulation before transitioning to supervised bedside practice, has any drawbacks.

The mastery-learning model is aligned with CBME in meaningful ways. First, the trainees’ ability to progress in their training is decided on the basis of predetermined outcome measures and levels of proficiency. Second, mastery learning includes several key features: trainee performance is assessed; a minimum passing score (MPS) is clearly identified; learning objectives, instruction methods, and assessments are aligned; and practice is continued until the MPS has been achieved. Third, variable duration of training is expected. Finally, the model assumes that everyone is capable of learning the skill through repetitive, deliberate practice (3). Compared with nonmastery models of training, SBML models are associated with improvement in skill development when performance is measured in simulation contexts, although, unsurprisingly, they also require more time for completion (4).

However, there are many drawbacks to SBML models, which we explore in this commentary: evidence in favor of SBML models is drawn from studies that examine one particular skill in isolation and ignore the feasibility of this model at a programmatic level; mastery is operationalized as an MPS on a simple checklist and does not take into account many other competencies required to skillfully manage a real-life scenario; waiting to reach mastery in simulation decreases time and opportunities for learning under supervision in real life; the initial artificial separation of simulation and real-life learning ignores elements of how transfer of learning occurs in different contexts; the model encourages a scripted approach rather than a flexible, adaptive approach, which is needed to manage the complexity and unpredictability of real-life situations.

Most studies that look at SBML focus on one particular skill and fail to examine the implications of SBML at a program level, at which many skills need to be learned, mastered, and applied. We therefore reviewed SBML studies that reported curricula time to teach procedural skills relevant to critical care (4–10). On the basis of the hours required to train to mastery reported in these studies, we calculated that each CCM trainee would need at least 23 hours of simulation-based procedural training alone before being allowed to participate in supervised practice (e.g., central venous catheters: 5 h, thoracentesis: 5 h, intubation: 3 h, arterial line catheters: 2 h and bronchoscopy: 8 h). Understandably, most of the skills in the studies and in most programs are taught in a “boot camp” fashion (condensed simulation practice that lasts many hours or the whole day) and ignore important elements of deliberate practice. Deliberate practice is defined as individualized training, designed by a teacher, to improve specific aspects of a performance through repetition. In addition, during deliberate practice, individuals need to monitor their training with full concentration, which is effortful and therefore limits the duration of other daily activities (11). SBML evidence has come from research in which extra resources were available, without evidence of sustainability once the study was completed. Consequently, programs do not often use SBML outside the research context because of logistical issues with organizing space, equipment, time for the trainee and instructor, and the ability to integrate the simulation activities in the overall curriculum.

The definition of “mastery” or “excellence” in SBML is also problematic. The main reasons to support SBML are to foster excellence in all learners and optimize patient safety. However, competence is operationalized as an MPS (or “standard set” in simulation), which is a checklist and/or global rating scale (GRS) developed by an expert panel that reflects a routine ideal scenario and does not capture many aspects of expertise. Learners may face significant challenges during the transition to real life because the context is different and requires judgment under pressure, managing consequences to patient care, and troubleshooting the unexpected, just to name a few challenges. It would be naive to believe that a trainee’s level of performance in simulation translates automatically to real clinical environments, especially in critical care. Correlation of performance in simulation and real life is nonlinear, as performance is context related. Extensive research conducted by Lave and Wenger (12) shows that our abilities are not applicable across highly variable contexts. Rather, new knowledge and skills are constructed in the course of understanding and participating in different situations, a process generally referred to as “situated learning.” It is not surprising that very few studies try to link SBML with patient outcomes. The studies that look at patients’ outcomes provide no data on the context of the procedure (elective, semiurgent, urgent), show association rather than causation, and compare SBML with didactic or apprenticeship training only (13, 14). The medical-education community should try to better understand the process of transfer from simulation to real life before embarking on outcome studies.

The SBML approach also disregards certain principles of transfer of learning. The requirement for competence in simulation before participating in practice and the many hours required to achieve that competence will most certainly lead to missed opportunities, during which trainees could learn under supervision—an effective contribution to development of expertise. Theories of transfer argue that transfer of skills depends on whether learners see similarities across different situations (15). Involving learners in real life helps them see the similarities and differences between simulation and bedside practice and primes them for better use of simulation practice subsequently. A combined approach simultaneously incorporating simulation and real-life learning would allow learners to compare and contrast simulation and real-life practice and may foster a more effective use of limited simulation resources.

Another weakness of SBML is that its description in the majority of studies and, consequently, its application by some educators is a fixed way of practicing a series of steps to achieve an MPS. Although SBML by definition does not specify a fixed way of teaching, the progression of training depends on having achieved a required competency (documented by reaching an MPS) before the next competency. Most assessment tools used to measure the MPS are based on a checklist or GRS that focuses on these steps. Although the competency in SBML could be “flexibility in managing a changing environment,” assessment tools to measure that competency are not fully developed. SBML then lends itself to teaching what can be measured as an MPS, which is most often a fixed way of learning a skill. In our rapidly evolving healthcare system, ambiguity, complexity, and novelty are frequently encountered and require clinicians to be flexible in their approaches. Mastery learning may lead to excellence in simulation-based performance, but it also encourages a “scripted” approach to procedural learning that may limit trainees’ ability to adapt to complex clinical situations, a feature of adaptive expertise (16).

Real critical-care situations are complex, and although SBML is a good concept with an important role in medical education, its implementation and integration with clinical learning may need further refinement. We end this commentary by providing a schematic of an organized, stepwise SBML curriculum (Figure 1) that readers can contrast with a schematic that illustrates the principles of an integrated curriculum (Figure 2). Simulations and clinical experiences share the same learning activities, objectives, and goals; however, the learning processes in simulations and clinical experiences are complementary but also different in terms of our cognitive and emotional learning. The complexity, dynamism, stakes, and human components cannot be easily replicated in simulations.

Figure 1.

Simulation-based mastery learning (SBML) central venous line (CVL) curriculum. Although we have selected CVL as a skill to illustrate an SBML curriculum, this model could be applied to any other skill. After the in-class (didactic portion of curriculum) and having successfully completed their knowledge test, the participants progress to simulation training. The simulation training initially focuses on procedural skills only (task trainer) then progresses to participants having to manage all aspects of CVL insertion (using simulated participants and a high-fidelity mannequin). After successful completion of simulation training as assessed by a checklist and a global rating scale, participants perform the procedures supervised at bedside. ICU = intensive care unit.

Figure 2.

Integrated curriculum. Simulations and clinical experiences are integrated, and they share the same objectives, goals, and learning activities. The learning processes in simulations and clinical experiences are complementary but also different in terms of cognitive and emotional learning. The complexity, dynamism, stakes, and human components cannot be easily replicated in simulations. There should be a parallel integration of both experiences, with one taking on more importance than the other, depending on learner and context.

Therefore, we advocate for parallel integration of simulations and clinical experiences in which, depending on the context and the learner, one might take on more importance than the other. Enforcing simulation performance standards to limit trainee engagement in real-life practice is artificial, relies unduly on simulation, and has potential drawbacks. Programs, while attempting to increase simulation training, should encourage participation in all opportunities (simulation and real life) and varied contexts, encourage problem-solving, and provide appropriate supervision and feedback to their trainees to foster medical expertise.