Impact of a Virtual Simulation-Based Educational Module on Managing Agitation for Medical Students.

Healthcare workers are at an elevated risk of experiencing workplace violence, which can lead to physical injuries as well as high rates of burnout and staff turnover [1, 2]. Therefore, managing patient agitation, defined as excessive psychomotor activity leading to aggressive and violent behavior, is a critical skill of physicians to help ensure the safety of patients and staff alike. Preventing violence requires the recognition and treatment of symptoms that can lead to agitated behavior as well as the employment of effective de-escalation techniques [3].

Learning to manage patient agitation can be challenging because it is a complex task. It requires an awareness of the environment, verbal de-escalation skills, establishment of rapport with behaviorally activated individuals, knowledge of potential etiologies underlying agitation, and efficient clinical assessment to develop an appropriate differential diagnosis. To help trainees develop this array of skills, educators need to impart knowledge regarding clinical assessment and diagnoses, in addition to communication and teamwork techniques. As with other skills-based learning, real-life clinical experiences and associated feedback is often preferred. However, instances of agitation present heterogeneously, unpredictably, and sporadically, especially within undergraduate medical education [4]. In vivo moments of agitation management may also not be safe for early learners given the risk of physical injury. Thus, early clinical exposure to agitation management is often limited for medical students. With such limited exposure, students may not be able to build confidence in working with agitated patients nor have opportunities to challenge preconceived misconceptions about increased risk of violence among psychiatric patients [5].

Formal training in behavioral emergencies is not standardized, particularly for medical students, and few targeted educational interventions have been studied. The limited literature available has shown that simulations are effective at improving confidence and competence among students and trainees [6-9]. In particular, simulated patient encounters involving agitated individuals can foster teamwork and help learners address the safety of both patients and staff [9, 10]. However, simulations can be resource-intensive, leading others to examine alternative methods for teaching and assessment such as videos and case vignettes [4, 11].

The onset of the COVID-19 pandemic disrupted medical education, creating the potential for significant educational gaps for medical students [12]. Many students had significantly less clinical exposure as on-site clinical clerkships at US medical schools were suspended in the early months of the pandemic. This diminished time in the hospital potentially widened an already existent educational gap in agitation management [13]. However, the pandemic also presented an opportunity for the integration of virtual technology into medical education and training [14, 15]. There is emerging evidence that virtual simulations can be used to address gaps in medical education [16]. Virtual simulation may be a unique tool to address students’ needs for education about agitation management.

We developed and piloted a virtual simulation-based educational intervention on agitation management for medical students. Our goal was to evaluate the feasibility of a virtual simulation format to prepare medical students for collaborative participation in managing agitation alongside house staff. To evaluate this educational intervention, we assessed participants’ awareness, confidence, and comfort with managing agitation. Here, we discuss the impact of the piloted intervention and the implications for developing future educational simulations on virtual platforms.

Session Design

We designed the virtual session on agitation management to fit within a two-hour block in the psychiatry clerkship curriculum. This session occurred using Zoom Videoconferencing technology (Zoom Video Communications, San Jose, CA). The session consisted of two segments, each containing a 30-min didactic lecture followed by two 15-min immersive virtual simulation scenarios. Groups of three volunteer students participated in each 15-min simulation activity and directly engaged the standardized patient while the remainder of the class observed. Each immersive simulation built upon the previous scenario, so that each group of three students implemented relevant skills at different steps throughout a single case from presentation to diagnosis. Table 1 provides an outline of the curriculum and progression of the simulations.

Table 1

Outline of curriculum and virtual simulation progression

		Nurse prompt	Standardized patient action/prompt	Learning objectives and ideal learner actions
Segment 1:	Lecture content (30 min)	Attending to physical safety; recognizing violence risk factors; verbal de-escalation techniques
	Simulation 1A (15 min): Physical safety and room choice	“Can you help me choose a room for a patient arriving shortly? EMS reports they have been agitated” *Give learners a brief triage summary of agitated patient prior to arrival	*Prompt with visuals of 3 possible rooms in the emergency room	*Recognize factors of a potentially risky patient and risky situation *Identify environmental interventions to mitigate risk
	Simulation 1B (15 min): Patient engagement and verbal de-escalation	“We need to help her calm down, I can’t even take her vitals”	*Patient placed in room chosen by prior volunteers and has a visible head laceration *Patient says “I don’t know why I need to be here. I’m perfectly fine and want to go home”	*Engage patient in a non-threatening way *Identify common ground with the patient and meet any possible psychosocial needs *Once calmer, help patient and nurse engage in taking vitals
Segment 2:	Lecture content (30 min)	Use of teamwork; appropriate use of emergency medications; differential diagnosis of agitation
	Simulation 2A (15 min): Psychopharmacology for agitation management	“She is demanding to leave, I really need help over here”	*Patient says “I’m not waiting around any more. I’m leaving” *As patient gets more upset and unable to be verbally de-escalated, states “I need something to help calm me down!”	*Engage patient in non-threatening, empathetic way *Identify when verbal de-escalation is no longer effective *Choose appropriate emergency medication for the situation; offer voluntary, oral medication if/when appropriate
	Simulation 2B (15 min): Differential diagnosis and workup of agitation	“Now that she’s calmer, what do you need me to do for this patient?”	*Patient cooperates with initial history *Says “I was drinking and must have gotten into a fight”	*Identify possible underlying etiologies for agitation *Obtain essential components of history *Appropriately identify initial lab tests and evaluation

The didactic content from the first segment of the curriculum was developed using previously published curricula and publicly available simulated agitation videos [17, 18]. In this initial segment, learners focused on attending to physical safety, recognizing risk factors for violence, and utilizing de-escalation techniques. The second segment, which was developed from previously published guidelines around behavioral health emergencies, focused on the use of teamwork and the appropriate use of emergency medications [19].

Simulations immediately followed each didactic segment and allowed students to practice skills discussed during the lecture. During the simulations, one of the facilitators played the confederate role of a nurse that helped students complete the scenario, while a second facilitator observed behind the scenes and directed the flow of the case. A trained actor played the standardized agitated patient, following pre-scripted prompts and conversing with the two facilitators through Google Hangouts (Google LLC, Mountain View, CA) instant massaging to receive instructions and appropriate responses as needed. At the end of the simulation, facilitators led a semi-structured debrief with all of the students that addressed motivations and decisions made by the student participants and reinforced key concepts relevant to each segment of the module. This tele-simulation was designed using previously published techniques and maintained as many of the cognitive and affective learning features of an in-person simulation as possible [20].

Pre- and post-test assessments were distributed to students via an anonymized survey to determine the degree to which the session impacted students’ perceptions on the importance of attending to safety and of agitation management as a skill. Pre- and post-session assessments also measured students’ concerns about working with psychiatric patients and their confidence in helping a team manage agitation. At the start of the two-hour block, all students were informed that the surveys were anonymous and voluntary. The students could participate in the virtual didactics session whether or not they completed the surveys, and their decision to participate or not in the survey did not impact their evaluation during the clerkship in any way. By completing the survey, they agreed to participate in data collection.

Each item on the survey was phrased as a statement, and students were asked to rate the statement on a 5-point Likert scale (from strongly disagree to strongly agree). Students were asked to complete the pre-test online prior to the start of the session and the post-test online immediately following the session. We used a paired t-test to compare pre-/post-session ratings for each question in the survey. This study was deemed non-human subjects research by the Yale University IRB. Statistical analysis was completed using SPSS statistical software, version 22.0 (IBM Corp).

Outcomes

A total of 64 medical students attended the lecture. The response rate for those who completed both pre- and post-session surveys was 67% (n = 43). Only the students who completed both surveys were included in analysis. Of those who responded, 46.5% identified as male and 53.5% identified as female. On the pre-survey, students on average disagreed with the idea that they had enough time dedicated to teaching agitation management in medical school (1.72 ± 0.59 out of 5).

Student perception of the importance of attending to safety did not appear to change between the pre-survey and the post-survey (from 4.5 ± 0.74 to 4.7 ± 0.71, p = 0.279). However, the recognition of managing agitation as an important skill increased from 4.5 ± 0.50 to 4.9 ± 0.35, p < 0.001. In addition, students’ confidence in helping manage agitation increased from 2.2 ± 0.75 to 3.2 ± 00.75, p < 0.001. Students’ worries about working with psychiatric patients decreased from 2.7 ± 0.87 to 2.2 ± 0.91, p < 0.001. Table 2 summarizes these results.

Table 2

Summary of pre- and post-session survey results

	Pre-surveyMean (SD)*	Post-surveyMean (SD)	p-value
Attending to safety is important in patient encounters	4.5 (0.74)	4.7 (0.71)	.279
Managing agitation is an important skill for healthcare providers	4.5 (0.50)	4.9 (0.35)	< .001
I am confident in my ability to help a team manage patients who are agitated	2.2 (0.75)	3.2 (0.75)	< .001
I worry about working with people with mental illness because they are more likely to become violent	2.7 (0.87)	2.2 (0.91)	< .001

*Each survey question consisted of agreement responses on a 5-point Likert scale (from 1 = strongly disagree to 5 = strongly agree)

Implications of the Educational Intervention

Overall, the decrease in worry, increase in recognition of agitation management as an important skill, and increase in confidence show the positive impact of a virtual simulation intervention to teach agitation management to medical students.

Prior to this session, medical student participants rated attending to safety as an important component of clinical encounters but did not feel they had adequate time dedicated to agitation management during their medical education. This suggests that there had been an unmet need within their medical education to learn about agitation management. Such a finding is consistent with a recent study that identified 25% of trainees who did not feel adequately prepared for medical practice by their medical education and named agitation as an area that was particularly relevant to their practice [13]. Such a gap is an opportunity for expanding access to agitation management education.

Our approach to target this gap was to use a virtual platform to deliver both lecture material and simulation components. The combined methodology proved to increase students’ recognition of agitation as an important skill and increase students’ confidence in helping their medical teams manage agitation. The increase in confidence is promising as it shows improvement of nearly a full point on the Likert scale. However, the rating is modest overall (3.2 out of 5), suggesting that the confidence remains proportional to the role and skill level of a medical student. Prior studies have shown similar improvement in confidence following in-person simulations [6-8]. However, this is the first to the authors’ knowledge to replicate an improvement in confidence utilizing virtual simulation.

Additionally, students reported a decrease in their concern about working with patients with mental illness. It is possible that this worry decreases for multiple reasons. Since worry was rated rather low prior to the intervention (2.7 out of 5), the information provided likely reinforced the fact that on the whole, individuals with mental illness are more likely to be the victims rather than the perpetrators of violence [5]. In addition, we saw an increase in students’ self-reported confidence. As confidence increases, it may have an inverse relationship with worry. Regardless of the driving cause, the improvement in students’ concerns about working with psychiatric patients is heartening.

This session was delivered at one institution by a single set of instructors, and the participants were medical students in a single clerkship class at one institution. As such, it is not known whether the findings are generalizable to medical students at other schools learning from other instructors.

Although there was a response rate of about two-thirds of the participants, it is possible that the voluntary nature of the survey questions self-selected for individuals who benefitted from the intervention. Zoom also presents a unique challenge in that those who are logged in, may not be fully engaged for the entire session. Lastly, the survey data presents self-reported information on attitudes and confidence, but it is not known whether the intervention changed knowledge or skills or if such changes led to better in vivo management of agitated patients.

It is promising to see that this technological format represents an effective teaching tool within undergraduate psychiatric education. Virtual formats of medical education have filled a crucial training need during the COVID-19 pandemic, but likely can have broader utility even after the pandemic wanes. Virtual simulation can also broaden the reach of simulation-based education by making them accessible to remote or lower-resourced areas that may not have simulation labs or psychiatric educators located on site. Virtual platforms can also create efficiency and lessen the resource burden of traditional simulation-based education. Virtual simulation has the potential to help address an unmet need by providing high quality, effective education on safety and agitation management skills for medical students as they embark on any of their clinical clerkships.