Cognitive load during training for out-of-department emergency responses.

Background: Emergency medicine (EM) physicians sometimes respond to critical events outside the emergency department. To prepare for these complex cases-typically called "rapid responses" (RRs)-EM residents receive simulation-based training involving four practice tasks and three exam tasks during a 1-day session. Cognitive load (CL) theory describes how humans function with limited working memories to perform complex tasks. RRs are expected to generate high levels of CL, but the profile of CL across providers and RR cases is not well understood. In this study, we analyzed resident's CL during RR training. We hypothesized variations in CL across individual and case and that exam cases would cause higher CLs than practice cases.
Methods: Residents anonymously self-reported CL levels after each case using the Paas scale, a single-item, 9-point scale from "very, very low CL" to "very, very high CL." To examine case-based differences in CL, data were rescaled by individual residents. "High CL" was defined as a score of 9/9.
Results: Among 18 residents participating, CLs ranged from 4 to 9, with median of 7 and interquartile range of 7-8. While many cases showed bell curve-like distributions of CLs, one case-a bleeding tracheostomy-showed a rightward skew reflecting higher levels of CL. No significant difference was found in CL between practice and exam cases. There were 20 reports (16.5%) of "high" CL with variation across residents (0/7 [0%] to 5/6 [83.3%] cases) and across cases (1/18 [5.6%) to 8/18 [44.4%]). Conclusions: The CL that EM residents experienced did show considerable interpersonal and intercase variation, but there was no significant difference between practice and exam cases. These results highlight several questions about how to optimally design future training, including how best to balance low and high CL training cases and which cases may require further training.

INTRODUCTION

In many hospitals, emergency medicine (EM) residents may be called to respond to a critical event like an airway emergency or cardiac arrest outside the emergency department. These rapid responses (RRs) are typically complex cases involving high‐acuity decision making in potentially unfamiliar environments. Cognitive load (CL) theory describes the ways in which our brains function with limited working memories to learn and perform tasks. , CL has been explored both in simulated and in real‐life scenarios, and differences in perceived CL have been used to facilitate structural‐design questions in emergency gear. , ,

RRs are challenging and are expected to generate high levels of CL, but the patterns of CL that EM residents may experience during different RRs are not fully understood. Crucially, understanding these patterns may help programs develop improved RR training for their residents. In this study, we analyzed the self‐reported CL that residents experience when undergoing mandatory, simulation‐based RR training. We hypothesized that CL would vary across both individuals and simulated cases and that exam cases would show higher CL than practice cases.

METHODS

During a single‐day training, residents received lectures and engaged in simulated cases including four practice cases (P1–P4) and three exam cases (E1–E3: difficult adult airway, pediatric arrythmia, bleeding tracheostomy). After each task, residents anonymously self‐reported their CL using the Paas scale, as shown in Figure S1. This scale asks individuals to rate CL along a 9‐point Likert scale ranging from “very, very low CL” to “very, very high CL.” Prior to their first practice station, residents received a brief introduction to CL theory, the use of the Paas scale, and the utility of measuring CL during training exercises. Additional details on study methods—including descriptions of the cases—are available as supplemental material accompanying the online article. This study was approved by the University of Southern California’s Institutional Review Board. Analysis was performed in Rstudio, extended by the tidyverse and ggridges packages. , , To examine case‐based differences in CL, data were rescaled by individual residents. Comparisons between tasks were tested by Wilcox rank‐sum testing, with alpha set to 0.05. “High CL” was defined as 9/9 on the Paas scale.

RESULTS

Of the 18 residents, 15 (83.3%) provided CL data on all seven cases, one (5.6%) provided data on six, and two (11.1%) provided data on five. Across all 121 observations, CL levels ranged from 4 to 9, with a median of 7 and interquartile range (IQR) of 7–8. Figure S2 shows the histogram of all CL scores. After rescaling by resident, CL ranged from –2.0 to 1.6 (median [IQR] 0.1 [–0.6 to 0.8]). Figure 1 shows the density estimates for CL scores by case across all residents. While many cases showed bell curve–like distributions, the third exam case (E3)—a pediatric arrythmia simulation—showed a rightward skew corresponding to higher levels of CL. No significant difference was found between practice and exam cases, with median (IQR) CL scores for practice of 0.13 (–0.64 to 0.59) compared to exam 0.00 (–0.49 to 0.95; p‐value ~0.33).

FIGURE 1

Distributions of CL Experienced Across Multiple Cases. Smoothed density estimates of the rescaled CL levels residents reported experiencing while performing practice (P) and exam (E) cases. Higher numbers and brighter colors correspond to higher levels of perceived CL. CL, cognitive load

There were 20 instances (16.5%) where residents reported high CL. Rates varied markedly across residents, from 0/7 (0%) to 5/6 (83.3%) cases (median [IQR] 14.3% [0%–28.6%]). Rates of high CL varied across cases, ranging from 1/18 (5.6%) to 8/18 (44.4%; median [IQR] 11.8% [8.7%–17.6%]). Figure S3 shows the breakdown of high CL, highlighting differences across cases and the high demand of E3.

DISCUSSION

We found considerable variability in CL both across different RR cases and among different residents, although generally CL demands were high, with residents frequently reporting maximum levels of CL. These high levels suggest that residents are often “maxed‐out” during this training and that educators might consider redesigning training for lower CL demands initially to allow for cognitive space to practice existing mental tools like stress inoculation or visualization. Additionally, the high levels of CL reported during the tracheostomy case might signal educators to provide more specific training in that area. We did not find a significant difference in the levels of CL between practice and exam cases. This could be due to our relatively low sample size or perhaps to baseline levels of high CL experienced over the course of the training day. Alternatively, different residents might show CL differently across formats of cases: for example, the group‐based format of the practice cases might generate higher CL for some than the one‐on‐one format of the exam cases.

LIMITATIONS

CL levels during a case may be influenced by factors outside the scope of the experiment: a resident who slept poorly the night before might experience a higher level of CL solely as a result of this. Since CL was reported anonymously, it was not possible to explore potential relationships between CL and demographic factors. While the residents were instructed to report CL after each case, some might have chosen to report levels at the end, potentially resulting in recall bias.

CONCLUSION

The levels of cognitive load that emergency medicine residents experienced during this intensive training showed considerable interpersonal and intercase variation but that there was no significant difference between practice and exam cases. These results support the use of cognitive load measurements during training for rapid responses and highlight several questions about how to optimally design training for intense and cognitively demanding scenarios. Future work exploring the interface between cognitive load and rapid response training might benefit from sophisticated quantitative metrics of cognitive load like pupillometry or heart rate variability as well as from qualitative metrics investigating trainee experience.

CONFLICT OF INTEREST

The authors have no potential conflicts to disclose.

AUTHOR CONTRIBUTIONS

Daniel A. Dworkis: Study concept and design, acquisition of the data, analysis and interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, statistical expertise. Daniel A. Dworkis takes responsibility for the manuscript as a whole. Aarti Jain: Study concept and design, acquisition of the data, analysis and interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content. Marissa Wolfe: Drafting of the manuscript, critical revision of the manuscript for important intellectual content. Stephen Sanko: Acquisition of the data. Sanjay Arora: Analysis and interpretation of the data, drafting of the manuscript, critical revision of the manuscript for important intellectual content.

Supplementary Material

Click here for additional data file.