Shaken and stirred: emotional state, cognitive load, and performance of junior residents in simulated resuscitation.

BACKGROUND: Patient resuscitation can be overwhelming for junior postgraduate medical residents due to its inherent complexity and high-stakes environment. Emotional states of unpleasant hyperarousal burden cognitive resources, contributing to cognitive overload and performance decline. Our objective is to characterize the associations between pre-scenario emotional state and junior residents' cognitive load and performance in a simulated-resuscitation, to provide evidence for informed curricular development.
METHODS: PGY-1 residents self-rated their emotional state before four simulated-resuscitation scenarios, and their cognitive load after. Faculty assessed performance with entrustment scores. Factor analysis identified the principal components of emotional state data. Linear regression models examined the relationship between pre-scenario emotional components, cognitive load, and performance scores.
RESULTS: 47/47 medical and surgical residents (100%) participated and completed Emotional State (99.5%) and Cognitive Load (98.9%) surveys. Positive invigoration and negative tranquility were the principal components. Pre-scenario tranquility was negatively associated with cognitive load (b= -0.23, p < 0.0001), and cognitive load was negatively associated with performance scores (b= -0.27, p < 0.0001). Pre-scenario invigoration was negatively associated with cognitive load (b=-0.18, p = 0.0001), and positively associated with performance scores (b= 0.08, p = 0.0193).
CONCLUSION: Amongst junior residents participating in simulated resuscitation scenarios, pre-scenario agitation (negative tranquility) is associated with increased cognitive load, which itself is associated with lower performance scores. These findings suggest residency programs should consider developing curriculum aimed at modulating residents' emotional agitation and reducing residents' cognitive burden to improve resuscitation performance.

Introduction

The resuscitation of acutely unwell patients can be an overwhelming experience for junior postgraduate medical residents.[1]–[4] Residents have identified emotional stress as an important contributor to their medical errors in the acute care environment.[5] Junior residents are especially susceptible to the effects of acute stress because of their lack of resuscitation knowledge and experience.[6],[7] Stress and emotional state have demonstrated influence on cognitive processing and performance, but the intricacies of this relationship remain unclear.[8]–[13]

The emotional state of learners affects cognitive processing and subsequent performance in the context of high-fidelity simulation amongst diverse groups.[8],[10],[11],[14] Emotion has two dimensions, valence – the extent which emotion is perceived to be pleasant or unpleasant – and arousal – the extent which emotion is experienced as activating or deactivating.[15]–[17] Amongst medical trainees, unpleasant emotions tend to encourage detail-orientated focus and reliance on familiar problem solving strategies, while pleasant emotions foster global processing and cognitive flexibility. Not only has affective and social cognitive research demonstrated the significant influence of emotion on information processing, decision making, and behavior,[18]–[20] but many have linked physicians’ emotional state to adverse clinical decisions and patient safety.[21]–[25] For example, negative emotions promote fast, automatic, and intuitive processing which predisposes clinicians to anchoring biases.[26],[27] Furthermore, low arousal states promote task disengagement by impeding executive function, while hyperarousal encourages tunnel-visioning and poor allocation of cognitive resources.[28]–[30] In fact, optimal arousal is unique to the individual’s experience, with inexperience predisposing to hyperarousal.[31] The increased burden of cognitive load is associated with high invigoration and low tranquility, two principal emotional elements found in trainees during simulation. [8],[10],[11]

Though originally described as a theory of learning, cognitive load theory (CLT) has started to be applied in clinical contexts, such as resuscitations.[32] Physicians are cognitively challenged by the complexity and emotional stress inherent to high-stakes resuscitations (intrinsic load) while also carrying the mental burden of off-task distractions (extraneous load).[32],[33] When cognitive load outweighs cognitive resources, clinicians are cognitively overloaded and performance declines. While intense emotions are known to influence mental acuity[4],[10],[34]–[36] –by interfering with cognitive processing and contributing directly to the burden of cognitive load[10],[32],[37],[38] – literature on the impact of physicians’ pre-scenario emotional state on cognition or clinical performance is sparse. We believe that faced with inherently stressful and difficult resuscitations, unpleasant hyperarousal burdens cognitive load and exposes physicians to performance pitfalls like anchoring biases and tunnel-vision. Junior residents do not yet have the mental schemas nor mitigation strategies to manage affect that practiced resuscitationists wield to handle the emotions and complexity of resuscitation.[4],[31],[39] To date, there has been limited research investigating the effect of residents’ emotional state prior to resuscitation scenarios and its implications on cognitive load and performance. Understanding the relationship between emotional state, cognitive load, and performance in the context of resuscitation will assist with the development of curricula that includes skills training to manage affect.

The primary objective of this study was to investigate the association between pre-scenario emotional state and self-reported intra-resuscitation cognitive load in junior residents participating in a summative resuscitation Objective Structured Clinical Examination (OSCE). The secondary objectives aimed to identify the principal components of pre-scenario emotional state associated with superior resident performance and to investigate the association between cognitive load and performance.

Methods

Setting and participants

We conducted an observational cohort study that took place during a summative simulation-based OSCE of the Nightmares Course at Queen’s University (Kingston, Canada) in December 2019.[40] All residents participating in the Nightmares Course OSCE were invited to participate (n = 47). The Nightmares Course is a longitudinal simulation-based curriculum in resuscitation medicine that includes 4 formative simulation-based sessions over a 4-month period. Following these, a multi-station summative OSCE takes place. Participants included junior postgraduate residents at the “foundations of discipline” level of training from 14 medical and surgical specialties. This level of training corresponds to a resident in their first postgraduate year, where a breadth of abilities is acquired before moving on to discipline-specific training. Written consent was obtained prior to the OSCE. Approval to perform this study was obtained through the Health Sciences Research Ethics Board at Queen’s University.

OSCE (Objective Structured Clinical Examination)

The OSCE consisted of four 8-minute simulated scenarios (ventricular tachycardia, bradycardia, upper gastro-intestinal bleed, and sepsis). Each scenario involved one or two Registered Nurses as confederates and a faculty assessor who assigned a performance-based entrustment score after observing each resident. Residents arrived in groups of four and rotated through each scenario individually (n = 188). Prior to each individual scenario, residents completed a pre-scenario Eight-item Emotional State Scale. Immediately following each scenario, residents completed the Cognitive Load Rating Scale.

Emotional state scale

The Eight-Item Emotion Scale (Appendix A),[8] was developed from the original semantic model of emotion and the refined circumplex model of emotion,[15],[16] and has been used in a similar context for medical and nursing students in simulation.[10],[11] This scale was used to measure participants’ pre-scenario emotional state and consisted of 8 bipolar oppositional descriptors of emotions, each on a 9-point Likert Scale from -2 to +2, with positive values indicating a more pleasant measure of emotion. These descriptors were: tense/calm, nervous/relaxed, stressed/serene, upset/contented, sad/happy, depressed/elated, lethargic/excited, and bored/alert.

Cognitive load rating scale

The Cognitive Load Rating Scale (Appendix A) has previous validity evidence as a subjective rating tool in several contexts,[8],[41] and was used to measure participants’ perceived cognitive load. Participants were asked to rate the amount of mental effort they had to invest during the previous simulation scenario, on a 9-point Likert Scale on which 1 = very, very small effort and 9 = very, very high effort.

Performance-based entrustment score

The Ottawa Surgical Competency Operating Room Evaluation (O-SCORE) was used to measure resident performance (Appendix A). It has validity evidence as a trainee evaluation tool,[42] and consists of a 1-5 scale with entrustment anchors varying from “1. I had to be there” to “5. I did not need to be there.”

Statistical analysis

Descriptive statistics (mean ± SD) were used to report residents’ rating of their pre-scenario emotional state and intra-scenario cognitive load. The reliability of the emotional state rating tool was assessed using Cronbach’s alpha coefficient and a coefficient of α > 0.7 was considered indicative of acceptable reliability. A Kaiser-Meyer-Olkin measure of sampling adequacy for the factor analysis produced an acceptable value of 0.78.[43] Exploratory factor analysis – which aims to identify unobserved factors that explain the variation in sets of observed variables – was performed on the individual items of the emotional state tool. Principal component analysis via orthogonal varimax rotation was used to extract major dispositional factors from participant’s emotion state prior to initiating resuscitation simulation, as this method distills large multi-dimensional datasets into few discreet factors for focused analysis, while retaining most of the data’s variation.[44] To ensure items with the highest loading had the largest effect on the factor score, we created a weighted sum score for the principal components of emotion by combining the emotion rating scale items that loaded on this component after multiplying each by its factor score.[45] To evaluate the association between emotional state and cognitive load, we used linear regression in which cognitive load was the outcome variable and the principal factors of emotion was the explanatory variable. To examine the association between measures of pre-scenario emotional state and performance, we used linear regression in which O-SCORE performance score was the outcome variable and the principal components of emotion was the explanatory variable. To study the relationship between cognitive load and performance, we used linear regression in which O-SCORE performance score was the outcome variable and cognitive load was the explanatory variable. Statistical analysis was conducted using the latest version of STATA.

Results

All forty-seven residents (100%) who participated in the Nightmares Course OSCE completed a total of 187 of 188 (99.5%) Emotional State and 186 of 188 (98.9%) Cognitive Load surveys. Residents represented 12 training programs (anesthesiology, general surgery, internal medicine, neurology, obstetrics and gynecology, ophthalmology, orthopedic surgery, pathology, physical medicine and rehabilitation, psychiatry, radiology and urology).

Pre-scenario emotional state

The subjective ratings of pre-scenario emotional state had both positive (alert, excited, elated, happy, contented) and negative (stressed, nervous, tense) components. Figure 1 illustrates the mean ratings for each emotional component on the Emotional Scale for residents prior to the first resuscitation simulation scenario.

Figure 1

Bar graph of the residents’ subjective rating of pre-scenario emotion reported prior to the first OSCE station.

Principal components of pre-scenario emotional state

The alpha coefficient for the Emotional scale was acceptable at 0.83.[46] The KMO statistic was 0.78. Eigenvalues >1 was set as the cut-off threshold for factor extraction. Two principal components (eigenvalues 3.76 and 1.32) were identified that explained 99.4% of overall variance. Rotated factor loading was performed. Using a rotated factor loading cut-off of |0.3|, three of the eight Emotional Scale items loaded on Factor 1 and the other five loaded on Factor 2. Factor loading scores are shown in Table 1. Factor 1 was composed of the emotional items of “(-)stressed/serene(+)”, “(-)nervous/relaxed(+)”, and “(-)tense/calm(+)”, reflecting the previously identified “tranquility” factor. Our residents’ ratings of emotion regarding this tranquility factor were negative and represented agitative unpleasant activation, yet this factor is still referred to by its positive bipolarity of tranquility by convention.

Table 1

Factor loading of principal components of pre-scenario emotional state, cut-off |0.3|

Emotional Bipolar Item	Factor 1 Tranquility	Factor 2 Invigoration
Bored/alert		0.50
Lethargic/excited		0.36
Depressed/elated		0.74
Sad/happy		0.81
Upset/contented		0.76
Stressed/serene	0.88
Nervous/relaxed	0.90
Tense/calm	0.87

The items (-)bored/alert(+), (-)lethargic/excited(+), (-)depressed/elated(+), (-)sad/happy(+), and (-)upset/contented(+) loaded onto Factor 2 with primarily pleasant activation ratings of emotion that align with previous literature’s “invigoration” factor. The residents’ ratings of emotion are presented in the context of the circumplex model of emotion and shaded by principal component in Figure 2. The mean ± SD factor loading scores for “tranquility” and “invigoration” were -1.3 ± 2.4 and 1.1 ± 1.9, respectively.

Figure 2

Radar graph of resident’s pre-scenario ratings of emotion superimposed on the circumplex model of emotions, shaded to identify the principal emotional components of negative tranquility and positive invigoration.

Pre-scenario emotional components and cognitive load

As illustrated in Figure 3, there was a negative association between pre-scenario tranquility and cognitive load (b=-0.23, 95% CI -0.29 – -0.16, r2 = 0.2, p < 0.0001), and a negative association between pre-scenario invigoration and cognitive load (b=-0.18, 95% CI -0.27 – -0.09, r2 = 0.08, p = 0.0001).

Figure 3

Scatterplots show the associations between A) Pre-scenario tranquility and cognitive load and B) Pre-scenario invigoration and cognitive load.

Pre-scenario emotional components and performance

As illustrated in Figure 4, there was no significant association between pre-scenario tranquility and performance (b = 0.05, 95% CI -0.01 – 0.11, r2 = 0.02, p = 0.0884), but there was a positive association between pre-scenario invigoration and performance (b=0.08, 95% CI 0.01 – 0.15, r2 = 0.03 p = 0.0193).

Figure 4

Scatterplots show the associations between A) Pre-scenario tranquility and performance-based entrustment score and B) Pre-scenario invigoration and performance-based entrustment score.

Cognitive load and performance

Figure 5 illustrates the linear regression analysis comparing cognitive load and performance that showed a negative association (b = -0.27, 95% CI -0.38 – -0.16, r2 = 0.12, p < 0.0001). The overall mean perceived cognitive load (n = 187) following each scenario was 6.42 (±1.44) on the 9-point scale, corresponding to a level between “rather high mental effort” and “high mental effort.” Grouped by OSCE scenario (n = 47), the mean cognitive load was 6.71 ± 1.42, 6.32 ± 1.36, 6.56 ± 1.41, and 6.06 ± 1.33, respectively. The overall (n = 188) mean O-SCORE was 3.59±0.97, corresponding to entrustment between “I had to prompt occasionally” and “I needed to be there just in case.” Grouped by OSCE scenario, the mean O-SCORE was 3.45 ± 0.79, 3.61 ± 0.88, 3.07 ± 0.96, and 4.20± 0.89 respectively. Table 2 provides descriptive statistics (mean±SD) of all outcomes measure.

Figure 5

Scatterplot shows the association between cognitive load and performance-based entrustment score.

Table 2

Descriptive statistics (mean±SD) of pre-scenario principal emotional components, cognitive load, and performance-based entrustment score, grouped by station (n = 47) and overall (n = 188)

	Station 1 (V-tachycardia)	Station 2 (Bradycardia)	Station 3 (UGI bleed)	Station 4 (Sepsis)	Overall
Tranquility	-1.94 ± 2.29	-1.02 ± 2.44	-1.20 ± 2.53	-1.06 ± 2.14	-1.30 ± 2.37
Invigoration	0.89 ± 1.91	1.25 ± 1.98	1.02 ± 1.94	1.26 ± 2.00	1.10 ± 1.95
Cognitive Load	6.71 ± 1.42	6.33 ±1.36	6.56 ±1.55	6.06 ± 1.34	6.42 ± 1.44
Entrustment Score	3.46 ± 0.80	3.61 ± 0.88	3.07 ± 0.97	4.20 ± 0.89	3.59 ± 0.97

Discussion

We have described the association between the pre-scenario emotional state of junior residents and their cognitive load and performance in a summative, simulation-based resuscitation OSCE. Our principal component analysis of pre-scenario emotional state data identified negative ‘tranquility’ (or agitation) and ‘invigoration’ as the most representative emotional factors. Increased pre-scenario agitation was associated with increased intra-scenario cognitive load. Increased pre-scenario invigoration was associated with decreased cognitive load and increased performance. Finally, we found increased intra-scenario cognitive load was associated with decreased performance.

Our findings of ‘tranquility’ and ‘invigoration’ as the two dominant emotional components is consistent with previous studies by Fraser (2012)[8] and Schlairet (2015),[11] which also used the Emotional State Scale. Tranquility has been reported to represent the psychomotor state of the participant, ranging from positive orientation (representing pleasant deactivation emotions of serene, relaxed, and calm) to the negative orientation (representing unpleasant activation emotions of tense, nervous, and stressed).[16] Whereas prior similar work has described tranquility in the positive orientation, our participants reported it in the negative – akin to agitation. Participants in our study completed the emotional state scale prior to the resuscitation OSCE scenario (pre-scenario), whereas previous studies have measured emotional state following simulation.[8],[11] The negative association between pre-scenario tranquility and cognitive load is also likely due to the timing of the emotional state survey as agitation prior to simulation may carry into the resuscitation experience and burden cognitive load. Alternatively, it may be that residents feel cognitively overloaded even before the start of the scenario, contributing to pre-scenario agitation. However, the pre-scenario emotional state survey cannot identify the cause of the underlying emotions. Identifying agitated residents may simply highlight those unprepared for the resuscitation OSCE, anxious about inter-resident competition, or stressed about their personal life. Regardless of the cause, this agitative affect is associated with residents’ cognitive load.

Considering the context of a summative simulation-based OSCE, we expected that junior residents would report high perceived cognitive loads. Previous research found similar cognitive load measures in first year medical students,[8] nursing students,[11] and medical staff,[36] in the simulated setting. According to CLT applied to the clinical context,[32] both affective stress and the inherent difficulty of the task contribute to intrinsic cognitive load. Our study kept the resuscitative task and presentation of information constant (intrinsic load) and minimized distractions from the physical environment (extraneous load), in order to observe the role of pre-scenario emotion in cognitive load and performance. Just as the emotional state survey does not identify the etiology of the agitation, Paas’ cognitive load scale does not discriminate between intrinsic and extraneous load. Nonetheless, we found that residents who reported increased pre-scenario agitation also reported increased overall intra-scenario cognitive load, possibly reflecting the impact of this emotion (a component of affect) on cognitive load. Furthermore, we found that a high reported intra-scenario cognitive load was associated with decreased performance. Cognitive overload is defined as the reduced ability to perform or make decisions when mental resources are overwhelmed by heavy cognitive load–described fittingly by fighter pilots as “helmet fire.”[47] Irrespective of whether residents’ heavy cognitive burden originates from the challenging resuscitation simulation itself (intrinsic load) or various secondary off-task reasons (extrinsic load), the state of cognitive overload is evident by the decline in performance–which clinically may result in adverse patient outcomes. Our finding that agitation is associated with increased cognitive load is in line with prior work in this field.[8],[11],[36],[48] Though the exact threshold of load is person specific, excessive emotional cognitive burden contributes to cognitive overload and poor clinical performance.[32] Despite this association, existing resuscitation curriculum focuses on medical knowledge and crisis resource management skills, and neglect than the emotional regulatory skills that may contribute to successful performance in resuscitation scenario.

The second principal component of residents’ pre-scenario emotional state was invigoration. This has also been identified by similar previous studies,[8],[10],[11],[14] and seems to be related to participants’ mood and motivation. Our study identified that more invigorated residents had decreased cognitive load and increased performance. Though it may be suggested that individual resident simulation interest and resuscitation preparation may contribute to these findings, further investigation into the nature of this potentially causal relationship is required.

This study has several limitations. First, the novelty of resuscitation and the simulation lab can generate cognitive load and burden cognitive capacity, yet this study did not control for prior simulation exposure, nor for clinical resuscitation experience. Second, when we attempt to measure complex constructs such as emotion and cognitive load, our results are limited by the bias and constraints inherent to a self-rating tool. Third, our results are not generalizable to senior residents and attending physicians, as experience is known to reduce intrinsic cognitive load (by automating schemas) and increase emotional intelligence, allowing concerted logical reasoning despite a high-stress environment.[31],[43]

Conclusion

In the context of a resuscitation-focused summative OSCE, we found that residents’ pre-scenario emotional state consisted of agitation and invigoration. Our findings show that high pre-scenario agitation is associated with increased cognitive load, and increased intra-scenario cognitive load is associated with decreased performance scores. Furthermore, increased pre-scenario invigoration is associated with decreased intra-scenario cognitive load and increased performance scores. Due to the observational nature of this study, it is not possible to conclude causal relationships. Future work is required to further define the associations between pre-scenario emotional state, cognitive load, and performance. Simulation-based resuscitation curriculum should be targeted at improving resident performance by providing teaching and experience to reduce its cognitive challenge. Traditional medical education focuses on teaching primary skills of resuscitation through algorithms, meanwhile, secondary skills to manage physicians’ mental state – such as pausing or deep breathing – are often considered afterthoughts. It is well established in education literature that teaching secondary skills independent of primary foundational teaching is destined for failure.[49] Rather, to reduce the cognitive challenge of resuscitation, learners should be holistically taught the psychological skills of resuscitation in concordance with its foundational knowledge. Future studies should explore the impact of stress mitigation strategies on cognitive load and clinical performance, with the goal of formal integration into resident training. As researchers unpack the complex human factors that affect mental capacity, post-graduate medical education should consider these findings and integrate psychological skills training to target residents’ emotional state, optimizing cognitive performance, and ideally improving patient care.