Optimizing Multidisciplinary Simulation in Medical School for Larger Groups: Role Assignment by Lottery and Guided Learning.

PURPOSE: Medical school simulations are often designed for a limited number of students to maximize engagement and learning. To ensure that all first-year medical students who wished to join had an opportunity to participate, we designed a novel method for larger groups. PATIENTS AND METHODS: We devised a low technology "Orchestra Leader's" chart approach to prominently display students' roles, chosen by lottery. During simulation, the chart was mounted on an intravenous pole and served as a group organizational tool. A course instructor prompted students using the chart to accomplish the course objectives in a logical order. Real-life cardiologists and gastroenterologists provided the students with expert subspecialty consultation. We analyzed 125 anonymous student evaluation ratings for 3 years (2017-2019) with a range of 8 to 19 students per laboratory session.
RESULTS: Our 2017-2019 larger group sessions were all rated as excellent (1.26, Mean, SD ±.510) on the Likert scale where 1.0 is excellent and 5.0 is poor. There were no statistically significant differences in overall ratings among the 2017, 2018 and 2019 sessions. The subspecialists were uniformly rated as excellent. Verbatim free-text responses demonstrated resounding student appreciation for the role assignment by lottery method.
CONCLUSION: We designed a novel, "Orchestra Leader's" chart approach for accommodating larger groups in a multidisciplinary simulation laboratory using role assignment by lottery, roles depicted on an organizational chart, and expert instructor prompting. Our consistently excellent ratings suggest that our methods are useful for achieving well-rated larger group simulation laboratories.

Introduction

High-fidelity medical simulation is known to be an effective teaching tool for medical students, particularly when the simulation engages the learner as an active participant.1–4 In order to maximize participation, simulation scenarios are traditionally designed for small student groups typically ranging from two to eight students.5–7

Historically, from 2009 to 2016, we offered four simulation laboratories at our medical school annually with a maximum of six students per session, up to 24 medical students each year. In these historic laboratories, one or two students may have played multiple roles while other students were more passive, contributing only marginally to the group learning. In 2017, our medical school issued a mandate that each simulation laboratory must accommodate every student who wished to participate rather than limiting the laboratories to only six students selected at random. When our first laboratory was conducted that year, on April 20, 2017, the laboratory was not well received, with student comments requesting more clearly defined roles and better organization. This feedback prompted us to develop our novel method and organizational system to accommodate the greater number of students who wished to join.

We responded by: (1) devising a low-technology, novel system of assigning student roles by random selection from an envelope, as in a lottery, (2) establishing an “Orchestra Leader’s Chart” to document each role and the student who chose it, and (3) incorporating a faculty facilitator who played the role of the emergency department nurse and prompted each student to perform his or her role in a logical clinical order. We hypothesized that our new system would create a positive student experience in the simulation laboratory, despite the large number of students, by giving each student a distinct and meaningful role.

Patients and Methods

We obtained IRB approval to collect student evaluations of our simulation laboratory for analysis on March 23, 2017 from Harvard Medical School.

History of the Case

This 60-minute simulation laboratory was originally created by SMK, the 2009 teaching fellow during the Gastrointestinal Pathophysiology Block with the help of EMH and HS.8 The Simulation Case was based on an unpublished interactive teaching case for second-year medical students during their Gastrointestinal Pathophysiology Block created in 1998 with an educational grant from Harvard Medical School (Shields HS, Shaffer K, Upton M. Harvard Medical School Intranet Website case for Gastrointestinal Pathophysiology Course, 1998). Over the ensuing years, from 2009 to 2016, the simulation laboratory was modified to incorporate multidisciplinary, real-life expert perspective by inviting cardiologists and gastroenterologists to be consultants in the exercise. An Attending Gastroenterologist (HS) acted as the Emergency Room Nurse from 2009–2016 and 2017–2019 for internal consistency. Beginning in 2017, the case was adapted for use with first-year medical students. From 2017 to 2019, all 60-minute simulation laboratories were conducted at the Neil and Elise Wallace STRATUS Center for Medical Simulation at Brigham and Women’s Hospital.

The current version of the simulation laboratory was designed to teach the basic principles of evaluation and triage. Gregory Smith, a 54-year-old male, presented with chest pain and anemia in the context of coronary artery disease, a coronary artery stent placed five years previously, and an actively bleeding gastric ulcer, likely exacerbated by the use of high-dose NSAIDs for knee arthritis in addition to chronic dual anti-platelet therapy for the coronary artery stent. (See , Harvard Medical School Case)

Number of Students

Our simulation laboratory began with each participating student choosing a role randomly out of a letter envelope. Each of the available roles (up to 21, as initially suggested by our simulation expert, JMR) focused on a distinct step in the evaluation of the patient (Table 1). The large number of available roles allowed us to tailor the role assignments to the number of participants who attended. If more than 21 students arrived to participate in a given session, two students could share a role. The roles were printed in large font on strips of paper, which were placed in a letter-sized envelope. Before the laboratory started, each student picked a role with the envelope held over his/her head by HS. All students watched as each student performed his or her role during the hour-long laboratory.

Table 1

Twenty-One (21) Individual Simulation Roles Made Available to Students, Grouped by Major Themes and Listed in Suggested Chronologic Order

Major Themes of Evaluation and Management	Individual Roles
History	History of present illness
	History of past illness
	Social History
	Family History
	History of Allergies
	Current Medications
	Scribe: Write down history
Physical exam	Report/interpret Vital Signs
	Physical Exam: Cardiac
	Physical Exam: Lung
	Physical Exam: Abdomen
	Evaluate Stool and Test for Blood
Objective data	Interpret Laboratory Data
	Interpret EKG
	Interpret Chest X-Ray
Differential	Differential Diagnosis: Cardiac
	Differential Diagnosis: Gastrointestinal
Management	Initiate Resuscitation Efforts
	Consult Cardiologist
	Consult Gastroenterologist
	Order Empiric Therapy in ED

“Orchestra Leader’s” Chart

Prior to the laboratory, the file folder was opened out and clipped to a standard clipboard. The available roles were glued using a glue stick onto name tags on the “Orchestra Leader’s” chart prior to the exercise, once the number of participating students had been determined. After roles were chosen by lottery, each student’s name was written by the Emergency Room (ER) Nurse on the Orchestra Leader’s chart under the appropriate role. The Orchestra Leader’s chart was then displayed on the IV pole next to the manikin (Laerdal SimMan 3G, Stavanger, Norway). Figure 1 depicts the Orchestra Leader’s chart with participant names during a mock laboratory using Stratus Simulation Center personnel to practice the method. In order to protect medical student confidentiality, no photographs were taken during the actual medical student laboratories.

Figure 1

Photograph of a mock simulation laboratory using the Orchestra Leader’s chart, with roles displayed, mounted on an IV pole at the head of the manikin and clearly visible to the entire group.

As the ER nurse called out each role and the student’s name, the student, who had picked the role, stepped up to play it. The Orchestra Leader’s chart served as an easily visible organizational tool during the simulation laboratory to guide all students through the performance of their roles in a logical clinical order. This was done to provide the students with a roadmap of the patient’s expected clinical evaluation and to keep the flow of teaching points well organized. Students were told before the laboratory started that each student would not be interrupted or have additions by other students to the role he/she was playing.

Simulation Laboratory Resources

The laboratory was conducted in a large simulation suite with a manikin that was designed to simulate a patient encounter in the ER. The simulation room was equipped with a whiteboard to record vital signs and relevant laboratory data and a video monitor to display supplementary educational materials including a PowerPoint slide set of endoscopic findings and pathology images. There was a wall phone that dialed into a faculty and simulation specialist area behind a one-way mirror so that students could call the cardiology and gastroenterology consultants and present the case to each of them in separate phone calls.

The voice and physical responses of the patient, Gregory Smith, were provided by a simulation specialist (SV) from behind a one-way mirror. The patient’s laboratory data indicated the presence of severe anemia with an elevated blood urea nitrogen level and normal creatinine suggestive of upper gastrointestinal bleeding. The patient’s EKG demonstrated 1–3 mm ST-segment depressions concerning for coronary ischemia.

Faculty Development for Subspecialty Consultants

Subspecialty consultation was provided by real-life cardiologists (FAJ, FLS, DMD, DED, ACS, MNY) and gastroenterologists (LFB, DXJ, ELB). Each faculty member was given key references pertaining to the case and decision making,9–12 a tour of the simulation laboratory, the Simulation Case, EKG, chest X-ray, and PowerPoint Slide set. All faculty were also trained in the role assignment by lottery method and the strategy to permit distinct, solo role appearances by each of the students. The consultants were encouraged to speak to the patient as if he were a real, live patient, examine his heart, lungs and abdomen with a stethoscope, and model collegial behavior with the other subspecialist in their discussions and decision making. The consultants were also encouraged to model collegial replies to the students about the indications for and timing of gastrointestinal endoscopic procedures and/or cardiac catheterization procedures, new medications to initiate in the Emergency Room, and transfer to the Cardiac Intensive Care Unit for monitoring and possible endoscopic procedure.

Statistical Analysis

Anonymous paper evaluations of the laboratory were collected over a three-year period (2017–2019). Evaluations were completed by the students at the end of each laboratory in a separate conference room without any simulation faculty present. Completion of the evaluation was voluntary. Forms were left face down and collected only after the students had left the room. The evaluation form asked students to rate the overall laboratory experience and the subspecialty consultants using a Likert scale (where 1 is excellent and 5 is poor) as well as provide free-text responses to two questions: 1. What did you like best about today’s session?; 2. Would you change anything about today’s session? (See Evaluation Form, ) This form was directly based on a previously published evaluation form used for medical students to rate a clinical communications exercise.13

An analysis of variance (ANOVA) was used to make comparisons between the anonymous student evaluations using the identical evaluation form for the years 2017–2019. For the purpose of analysis, laboratory sessions were organized by the number of student participants into three groups (8–10 students, 11 students, 12 to 19 students) of roughly equal size (n=46, n=33, n=46, respectively). Because no comparable evaluations had been requested from the students in the 2009–2016 laboratories, no historical comparisons could be made. Analyses were performed (SP) for the overall rating of each simulation laboratory and individual ratings of the cardiology and gastroenterology consultants using IBM SPSS Statistics (V26).

Results

Over the three-year period of this study from 2017 to 2019, 125 students (2017, n=33, 2018, n=39, 2019, n=53) evaluated a total of eleven (11) simulation laboratories. The number of students ranged from 8 to 19 per laboratory. As noted in the Introduction, the first laboratory on April 20, 2017 did not use the “Orchestra Leader’s” chart as the organizational tool and was excluded from analyses. The average number of students was almost twice the usual six students (prior to 2017) at 11.36 students per laboratory. Evaluations were completed by all but two students who had to leave the laboratory early (125/127) for a 98.4% completion rate.

The laboratory evaluations were uniformly excellent (Likert <1.5) with an overall mean rating of 1.26 (SD±0.510) for 125 students’ anonymous evaluations (Table 2) over 3 years, 2017, 2018 and 2019 (see ). ANOVA analysis demonstrated no significant difference among years (Table 2). Notably, the overall mean score for 2019 of 1.38 (SD±0.627) was in the excellent range on the Likert scale even though in 2019 we had the largest group of 19 students recorded for one laboratory. The single largest laboratory session with 19 students received an overall rating of 1.47 (SD±0.612). In this largest laboratory, both the cardiologist and the gastroenterologist received perfect mean scores of 1.00 from all 19 students.

Table 2

Analysis of Variance (ANOVA) Comparison of Mean (SD) Likert Scalea Ratings Given by Medical Students for Each Year of the Large Group Multidisciplinary Simulation Laboratory Experience

	n	Overall Assessment	GI Instructor	Cardiac Instructor
2017	33	1.18 (0.392)	1.00 (0.000)	1.00 (0.000)
2018	39	1.18 (0.389)	1.00 (0.000)	1.08 (0.269)
2019	53	1.38 (0.627)	1.06 (0.233)	1.07 (0.267)
TOTAL	125	1.26 (0.510)	1.02 (0.154)	1.06 (0.231)
		p=0.103b	p=0.269b	p=0.126b

Notes: aLikert scale used where excellent =1 and poor =5. bp-values for the ANOVA comparison between yearly mean Likert scale ratings in each category. A p-value of >0.05 indicates that there is no statistical difference between the scores for each year.

A histogram plot of the number of students participating versus the overall rating indicated that the mean overall rating was optimal for laboratories with 8 to 10 students (mean overall rating 1.15, SD±0.015) (n=46 students, five groups total) and was virtually the same for groups of 11 students (mean overall rating 1.18, SD±0.392) (n=33, three groups total) (Figure 2). Groups of 12 to 19 students (n=46, three groups total) evaluated the laboratory less well, although the mean overall rating was still excellent, <1.5 on our Likert scale (mean overall rating 1.43, SD±0.655) (Figure 2). An Analysis of Variance yielded a p-value of 0.015, and Bonferroni corrections indicated that the mean overall score for the 8 to 10 student groups differed significantly from that of the 12 to 19 student groups (p=0.022) (Figure 2), but not from the groups with 11 students.

Figure 2

Overall mean rating by Likerta scale of the laboratory experience by the size of the medical student group.

The sub-specialty consultants’ ratings were uniformly excellent. The gastroenterology consultants’ mean ratings ranged from 1.00 to 1.07 over the eleven laboratories over the 3 years (p=0.072) (see ) and the cardiology consultants' mean ratings ranged from 1.04 to 1.07 (p=0.760) (see ). There were no significant differences between the ratings of the gastroenterologists and cardiologists over the 3 years (p=0.349)

The students were asked on the evaluation form what they liked best about the laboratory. As seen in the anonymous verbatim free-text comments in Table 3, the students were enthusiastic and appreciative of having an a priori clearly defined role that allowed everyone to participate.

Table 3

Verbatim Anonymous Medical Student Comments Regarding Large Group Multidisciplinary Simulation Laboratories

2017	“I Liked That Everyone Had a Discrete Role and Responsibility”
	“Loved that we had assigned roles and were kept to our roles.”
	“I liked being given clear roles and for the session being structured by the specific roles … it made the session less overwhelming”
2018	“Very effective way to tie class concepts together!”
	“I liked best the clear roles.”
	“I liked that we were active players in the unfolding of a scene … great job of making this feel low stress”
2019	“Opportunity for all students to be involved!”
	“Very hands-on, interactive …. clear roles”
	“It was clear what each person’s role was and what we were doing next … no confusion.”

In addition, students enjoyed interacting with the sub-specialty consultants as colleagues and watching the consultants’ model collaborative cross-disciplinary discussions (Table 4).

Table 4

Verbatim Anonymous Medical Student Comments Regarding Subspecialist Consultants from 2017 to 2019

“I Appreciated How the Gastroenterologist and Cardiologist Approached This Sim as Though it Were a Real Patient.”

“Interacting with the gastroenterologist and cardiologist was so much fun; they treated us as colleagues but were also teaching. I look forward to such collaboration in the future!”

“Helped us reason through the case and pathophysiology”

“Great didactic learning and questions in the moment”

“Phenomenal instructor who was excellent at answering questions.”

“The ability to interact with real attendings as if they were our equals.”

“Combined cardio and GI integration to see how the two disciplines have to consider each other.”

Discussion

The opportunity to participate in simulation laboratories can greatly enhance the educational experience of medical students1–4 but has been traditionally limited in size due to practical considerations of laboratory space and resources. From 2017 onward, we successfully met our medical school’s challenge of adapting our existing small-group simulation curriculum with a maximum of 6 students for use with larger student groups, up to 19 students at one time, using the same laboratory space and resources.

Once we received our medical school mandate to include all students who wished to participate, we recognized that the educational method had to be flexible, as student groups might fluctuate in size on short notice based on student attendance. We accomplished the goals of providing a distinct and meaningful participatory role for each student in the simulation exercise while maintaining the flexibility to tailor the number of distinct roles at the start of each session by employing a simple yet novel method of the role assignment by lottery and an “Orchestra Leader’s” chart to coordinate the roles throughout the exercise with expert instructor prompting.

Other simulation research studies and evidence-based reviews have demonstrated significant added value for simulation-based education for medical students and residents-in-training over traditional clinical medical education.1–4 Similar to real-life Code Blue scenarios in the hospital, having too many people in the room during simulation exercises may lead to confusion and negatively impact the outcome. Due to this perceived limitation, simulation scenarios for medical learners are often designed to accommodate only a limited number of participants.5–7

To our knowledge, this is the first paper to describe a specific, inexpensive, straightforward method to accommodate larger medical student groups successfully in simulation learning. We based our strategy to engage each student in a distinct and active role on our prior medical school experiences with Communication Skills Exercises where Harvard-MIT students asked that each student has an active role.13 Establishing 21 roles provided the flexibility to expand the number of participants to 21 or more, with 2 students sharing a single role if necessary, or to contract to the specific number of participants by asking students to play more than one role, particularly in the area of history taking or physical examination.

Our strategy to assign each student to an active rather than observational role was also supported by Reime et al’s 2017 study of emergency medicine simulation learning.14 The authors found that participants and observers had similar scores in predefined learning outcomes, but the qualitative data indicated that students preferred the participant roles for the purposes of inter-professional learning and building confidence.14 In 2018, Blanie et al provided evidence of improved learning outcomes for students participating in an anesthesia simulation program when compared to students only observing.15 Robles et al also showed that students learning about delirium through watching a theatrical performance with trained actors demonstrated higher learning scores as compared to those watching a video.16

In terms of the optimal number of students for simulation laboratories, Rezmer et al previously reported on their experience teaching a Code Blue simulation to medical students in groups of two, three and four.7 No difference in medical student success or impression of Code Blue simulation exercises was noted in varying group sizes of two to four students.7 Mahling et al reached a similar conclusion in their prospective, randomized study examining Basic Life Support for medical students.5 Their findings support the fact that groups of three, five or eight students can be effectively taught with similar outcomes.5 Similarly, Miloslavsky et al designed resident simulation studies to include no more than up to eight resident participants at a time.6

In addition to accommodating a greater number of students than has been previously reported, we designed a simple organizational tool. We established the “Orchestra Leader’s” chart, listing roles in the logical clinical order. As each student chose a role by a random lottery process, the Emergency Room (ER) Nurse wrote each student’s name on the chart in the appropriate space. This simple clipboard was mounted prominently, in full view of all students and faculty, on the IV pole next to the manikin to keep track of the correct chronological sequence of roles.

The success of our method was evident in the consistently excellent ratings over eleven laboratories over a three-year span of first-year students. While we do not have historical ratings for the 2009–2016 simulation laboratories for comparison, the overall group ratings for 2017, 2018 and 2019 were consistently in the excellent range. Anonymous student comments reflected their appreciation for having a defined role in the simulation exercise. The cardiologists and gastroenterologists were praised for the high educational value of their explanations, knowledge, teaching skills, collaboration, collegiality and how realistically they approached the manikin as a live patient and the medical students as valued colleagues. We recognize that this method might be applicable and useful for resident physicians and attendings, as well as other medical provider groups such as registered nurses, nurse practitioners, and certified registered nurse anesthetists.

Limitations

We acknowledge that a major limitation of our study was the lack of a medical student control group against which we could compare the success of our method. However, our method was developed in real-time to accommodate the changing demands of our medical school administration that all students be given the opportunity to participate in this optional activity. The creation of a control group requiring a separate IRB application and approval process was not practical given that the laboratories had to be completed over a relatively short time frame on the medical school calendar.

Our method is not suitable for simulations designed to teach procedural skills where individual repetition is essential. The durability and permanence of what was learned in larger groups compared to smaller groups also remains unclear and could be studied in the future. Lastly, it is important to note that our evaluation form was not designed specifically to assess the novel method, but rather the overall medical student learning experience.

Conclusion

In summary, we have presented an innovative, straightforward, and low-technology, but successful, method for adapting small-group simulation curricula for use with larger groups (in our experience, up to 19 students). The key elements of our method included role assignment by lottery, an “Orchestra Leader’s” chart, expert instructor prompting and real-life subspecialty consultants who integrated the concepts for the students and were approachable and collegial. The strengths of our method included its generalizability to a wide range of simulation curricula, except for teaching procedural skills, and the adaptability of the method to different size groups. In addition, our method did not require any additional laboratory space or resources above that which we had previously employed for our smaller laboratories of six students before 2017. In contrast to many emerging simulation learning strategies, our method also did not require expensive technologies to implement. We believe that our novel and practical organizational method can serve as a valuable strategy to maximize access to simulation and high-yield learning for early-stage medical students.