"Flipped" clinical rotations: A novel approach.

BACKGROUND: Near the beginning of the COVID-19 pandemic in the United States, medical students were pulled out of all in-person patient care activities. This resulted in massive disruption to the required clinical rotations (clerkships), necessitating creative curricular solutions to ensure continued education for medical students. APPROACH: In response to the lockout, our school adopted a "flipped" clinical rotations model that assigned students to remote learning activities prior to in-person patient care activities. This approach allowed students to continue their clinical education virtually with a focus on knowledge for practice while awaiting return to the shortened in-person portions of their rotation. In planning the modified clinical curriculum, educational leaders adhered to several guiding principles including ensuring flexible remote curricular components that would engage students in active learning, designating that no rotation would be completely virtual, and completing virtual educational activities and standardised exams before students returned to in-person experiences. EVALUATION: End of rotation evaluations and standardised exam scores were analysed to determine the effectiveness of this model. Despite the disruption associated with the pandemic and the rapid implementation of the "flipped" rotations, students continued to rate the overall experiences as highly as traditional clinical rotations. Students also performed similarly on standardised exams when compared to cohorts from other classes at the same experience level. IMPLICATIONS: While borne out of necessity during a pandemic, the lessons learned from our implementation of a "flipped" rotations model can be applied to address problems of capacity and clinical preparedness in the clinical setting.

BACKGROUND

In March 2020, in response to the emerging COVID‐19 pandemic, the Association of American Medical Colleges (AAMC) issued guidance suspending medical students from all direct in‐person patient care. While this resulted in massive disruption to required clinical rotations (i.e., clerkships), it also presented opportunities for innovative curricular solutions that could transform medical education, by leveraging existing digital infrastructure and optimising “flipped” experiences. , , ,

The “flipped” classroom model has been widely and successfully adapted in medical education, particularly in the preclinical curriculum. Research has shown that the “flipped” classroom has several potential advantages, including its ability to create time and space in an existing curriculum for educational innovations. The “flipped” classroom model has also been used on a limited basis for conference topics within some clinical rotations, to save time and improve student engagement. , , , To meet the challenges of the recent pandemic, we adopted a model of delivering clinical knowledge “en bloc” in advance of in‐person activities across all clinical rotations. Our experience suggests that this model may have utility in more conventional educational contexts.

APPROACH

Case Western Reserve University School of Medicine (CWRU SOM) responded to the AAMC guidance by creating a “flipped” clinical rotation model that was applied across all core clinical rotations: family medicine, internal medicine, neurology, obstetrics and gynaecology, paediatrics, psychiatry, surgery, emergency medicine, and geriatrics. Clinical elective rotations, including acting internships, were not “flipped.” Traditionally, students rotate through clinical rotations with direct patient care activities fully integrated with teaching sessions (lectures, case‐based discussions, small group learning) and culminating in a standardised exam. Our “flipped” clinical rotations model assigned students to virtual knowledge‐building activities followed by a standardised exam prior to a briefer period of in‐person patient care activities. This approach allowed students to continue their clinical education virtually with a focus on “knowledge for practice,” while awaiting return to the shortened in‐person patient care portions of their rotations.

This approach allowed students to continue their clinical education virtually with a focus on “knowledge for practice.”

Two hundred ten medical students were removed from their clinical rotations immediately following AAMC's guidance. Educational leaders worked quickly to develop a modified curriculum that could be applied during the suspension from clinical activities. While initial guidance from accrediting organisations suggested that it may be reasonable to delay or cancel certain clinical rotations due to the pandemic, we opted to ensure that every student would experience all of the required rotations. In planning the modified clinical curriculum, educational leaders adhered to several guiding principles: (1) ensure a flexible virtual curriculum that would engage students in active learning and knowledge‐building; (2) all rotations would require hands‐on patient care activities (shortened up to 50%); (3) virtual educational activities (e.g., lectures and conferences) would occur prior to in‐person clinical experiences; and (4) standardised exams (e.g., National Board of Medical Examiners Subject exams) would be given before students were reintegrated into the corresponding clinical work. Rotation directors across our four hospital affiliates collaborated by discipline to create virtual curricula that mapped to the university's existing rotation‐specific learning objectives. Virtual curricula included teaching sessions led by faculty, residents, and graduating medical students, interactive online cases (e.g., Aquifer, OnlineMedEd), , and some virtual health care visits.

In addition to developing content and structure for the virtual curriculum, educators adapted schedules to accommodate the initial and any subsequent clinical suspensions. Educators avoided extending the year into the following academic year, to prevent adverse impacts on graduation or residency applications, and rotation start dates for subsequent classes. As the initial surge of COVID cases waned locally, a restart date of 1 June 2020, was identified as the earliest day that students would be safely allowed back into the hospitals (example revised schedule shown in Figure 1). While the total length of each clinical rotation was not shortened, the in‐person portions were, with time spent on the virtual curricula making up the difference. This allowed students near the end of the academic year (late‐year) to finish without creating delays for the rising class (early‐year) students and avoided the need to teach two cohorts at the same time. Early‐year students participated in the virtual curriculum while late‐year students completed in‐person clinical activities.

FIGURE 1

Traditional and “flipped” schedules for a late‐year student that still needed to complete neurology, psychiatry, surgery, and emergency medicine rotations; 1 June 2020 was chosen as the in person restart date

This allowed students near the end of the academic year (late‐year) to finish without creating delays for the rising class.

All teaching sessions and test preparation for the standardised exams were moved to the virtual phase of the rotations. In our traditional model, students take standardised exams at the end of the rotation, requiring them to divide their time between patient care activities and exam preparation. Scheduling the exams before in‐person activities allowed students to concentrate on patient‐centred activities upon their return to the clinical setting. Also, days normally reserved for exam administration were reclaimed for clinical activities.

EVALUATION

In‐person clinical activities resumed on 1 June 2020 at all affiliate hospitals. We measured the success of the “flipped” clinical rotations by reviewing qualitative and quantitative data from the end‐of‐rotation evaluations completed anonymously by students and by student performance on standardised exams. Quantitative and qualitative data were obtained from CWRU SOM's Medical Education Data Registry, which is an IRB approved data registry of aggregated, de‐identified data (IRB20151105) that can be used for educational research and quality improvement purposes. All “flipped” rotations pivoted to pass/fail (in lieu of tiered grading), so final grade distributions could not be analysed to detect grade differences with traditional rotations.

Student ratings of the perceived quality of each rotation were examined for comparison between traditional and “flipped” rotations using a rating scale of “Poor, Fair, Average, Very Good, or Excellent.” The traditional cohort were students from the Class of 2021 who had undertaken rotations that ended before March of 2020. The “flipped” data come from the same class of students completing evaluations for rotations that started in March 2020 or later and ended by 24 July 2020. A total of 1354 ratings were collected, 927 from traditional rotations and 427 from “flipped” rotations. The distribution of ratings from all rotations combined is shown in Figure 2a.

FIGURE 2

(a) Distribution of ratings from all rotations combined. (b) Distribution of ratings by discipline for traditional and “flipped” rotations with associated p‐values using a Pearson chi‐square asymptotic two‐sided analysis. Abbreviations: EM, emergency medicine; FM, family medicine; IM, internal medicine

Our institution uses the metric of % Very Good or Excellent in evaluating each rotation, with a target rating of >80%. The distribution of overall ratings by discipline for traditional and “flipped” rotations are shown in Figure 2b. There was no statistically significant difference between traditional or “flipped” rotations in any discipline (Pearson Chi‐Square asymptotic two‐sided analysis [SPSS 26]).

Qualitative feedback was examined looking for comments regarding the virtual curriculum and shortened in‐person rotations. Forty‐five student comments relating to the altered experience were collected, of which 27 were positive and 18 were negative or had suggestions for improvement. (Table 1) Positive comments included appreciation for the rapid response to create new curricula and faculty engagement with education. Negative or constructive comments included challenges of the shortened schedule and limited patient exposure. Additionally, anecdotal reports from rotation directors and faculty suggested that students on “flipped” rotations were better able to concentrate on patient care activities and less distracted by preparation for exams; and that many demonstrated an expanded knowledge base. Rotation directors also reported challenges associated with implementing a virtual curriculum during a time of increased stress.

TABLE 1

Representative comments from end of rotation anonymous surveys

Positive comments

Negative or constructive comments

Appreciation for rapid response and flexibility: “Fantastic learning experience given unique challenges that came with COVID and shortened clinical rotations. Gave a great deal of autonomy for students to learn and practice medicine.” “Did a fantastic job with being flexible given unique circumstances of COVID and shortened clinical rotations.” “Excellent overall. Great adapting to COVID to ensure a quality educational experience.” “The flipped classroom approach to virtual curriculum was fantastic ‐ helped solidify concepts and engage students. Clerkship directors were responsive to student concerns and were flexible in approach to teaching to best meet students' needs. All staff and residents were strong teachers during the in‐person clerkship.” Appreciation for faculty engagement: “Very detailed and organized in terms of accommodating students during the height of the COVID outbreak and planning meaningful online activities. Also, very interactive faculty and residents that are interested in teaching students.” “The residents and faculty were very much engaged in our learning and enthusiastic to give us as much experience as possible in the short amount of time we had.” “The clinical experience was excellent. The residents were all wonderful to work with and very supportive of us. Clerkship directors responded to feedback regarding the virtual curriculum.”

Challenges of shortened schedule: “It would be nice if the clerkship were longer, but this was limited due to COVID‐19.” “Unfortunately, just due to the nature of the shortened rotation from COVID and virtual visits, I was only able to see 1 patient per day, most of which were healthy elderly patients that did not have any ‘geriatric syndromes’. I can only rate the clerkship as fair.” Limited patient exposure: “It was hard to only do a lot of televisits, but I know given the pandemic, that was the only option. Maybe doing teaching via conferences as the televisits were not that educational would be useful.” “Not sure how to improve this, but patient volume on the inpatient service and clinic were pretty low.” “Virtual curriculum Friday lectures were so long (9 am ‐ 5/6 pm) that I could not pay attention to them. They should have been reduced or spaced out. It was not effective learning because we were all so tired.” “I liked our teaching attending sessions but I do not know how truly helpful they were all the time and since we had done a virtual didactic portion I sort of wanted to stay with the team in the afternoon.”

Students on “flipped” rotations were better able to concentrate on patient care activities and less distracted by preparation for exams.

Results from the standardised exams were analysed using one‐way analysis of variance (ANOVA) to determine if there were any differences between traditional and “flipped” rotations (Table 2). To compare students taking “flipped” rotations with peers undertaking traditional rotations, exam scores were analysed for “late‐year” M3 students from two consecutive academic years at the same time point in the academic cycle (Spring 2019 vs. 2020). The exam performances of the two groups were statistically similar. Students in academic year 2019–2020 who transitioned from traditional rotations to the “flipped” rotation model achieved better test scores near the end of the year (with statistical significance in some disciplines), a pattern consistent with past experiences at our school.

TABLE 2

Two comparisons of student performance on standardised exams

Discipline	Traditional Late‐year Learners Spring 2019 Mean Score (N)	“Flipped” Late‐year Learners Spring 2020 Mean Score (N)	ANOVA P value	Traditional Early‐year Learners Summer 2019 Mean Score (N)	“Flipped” Late‐year Learners Spring 2020 Mean Score (N)	ANOVA P value
FM	73.6 (51)	80.7 (45)	<0.001	74.2 (50)	80.7 (45)	<0.001
IM	75.0 (67)	77.6 (48)	0.158	72.7 (53)	77.6 (48)	0.023
Neuro	80.0 (64)	81.0 (67)	0.473	80.7 (25)	81.0 (67)	0.872
Ob/Gyn	77.1 (55)	78.6 (54)	0.266	74.3 (58)	78.6 (54)	0.14
Paediatrics	79.5 (50)	77.0 (53)	0.107	72.7 (58)	77.0 (53)	0.025
Psych	82.4 (64)	84.1 (69)	0.129	81.5 (26)	84.1 (69)	0.106
Surgery	76.8 (68)	74.7 (56)	0.168	73.1 (20)	74.7 (56)	0.478

Note: (1) Late‐year learners from two classes. (2) Early‐ vs. Late‐year learners from the same class.

Abbreviations: EM, emergency medicine; FM, family medicine; IM, internal medicine.

IMPLICATIONS

The “flipped classroom” model has been successfully integrated into a variety of areas of the medical education curriculum. Individual rotations that have adapted this model have reported increased learning motivation, improved interactivity during teaching sessions, and better post‐test performance. , When implemented across an entire clinical rotation program, we showed that overall ratings and student performance remained similar to our traditional approach.

Our “flipped” rotation model aided in re‐integrating students into the clinical setting after a pandemic lockout by delivering clinical knowledge ahead of time. Furthermore, it allowed medical students to complete core clinical training without experiencing significant delays. While the pandemic removed students from in‐person activities for 11 weeks, all students were able to complete core rotations on schedule (or with no more than a 1‐month delay) without shortening the total length (remote learning plus in‐person activities) of each rotation. This approach allowed most students to maintain their original sequence of rotations in the critical time between the end of their first clinical year and the residency application season.

It allowed medical students to complete core clinical training without experiencing significant delays.

Comparing traditional and “flipped” rotations, we did not detect any statistically significant difference in the ratings of the overall quality of our rotations. Similarly, we did not detect an adverse impact on standardised knowledge test scores. This was a reassuring finding as it was uncertain at the outset whether exam performance would suffer in the absence of clinical reinforcement of knowledge. However, we could not compare clinical competence and grade distributions with traditional rotations as our usual tiered grading was suspended during the “flipped” rotations.

Use of the “flipped” rotation model has several potential applications beyond the pandemic. It can help shorten required rotations to facilitate schedule shifts of the clinical year: many medical schools are moving clinical rotations earlier in the curriculum to allow students more time for clinical electives, career exploration, or to accommodate board exams after clinical training. , During such transitions, there is often a transient increase in demand for clinical training spots as a new class of students starts earlier, overlapping with the previous class finishing. Decoupling the knowledge for practice and in‐person clinical training allows one group of students to train in‐person while the other engages in remote learning and permits a greater number of students to engage in clinical activities simultaneously without compromising student learning.

Decoupling the knowledge for practice and in‐person clinical training allows one group of students to train in‐person while the other engages in remote learning.

Other medical school efforts or circumstances that require increased clinical training capacity can benefit from using the “flipped” rotations model including increased student enrolment, curricular reforms that result in crowding in the clinical space, or loss of clinical sites. “Flipped” rotations may also be valuable at the beginning of the first clinical year when students may struggle with the transition into the clinical setting. By delivering the knowledge for practice ahead of their first in‐person experiences, students can augment their learning during their first clinical rotations and build confidence. ,

By delivering the knowledge for practice ahead of their first in‐person experiences, students can augment their learning.

At our institution, we reverted to the traditional rotation structure after the initial disruption and had no subsequent suspensions despite additional surges caused by coronavirus variants. Nonetheless, the experience and data gained during this period allows us to better plan for an upcoming forward shift in the clinical rotations schedule that will require transiently shortened rotations given capacity limitations.

In summary, a “flipped” clinical rotation model with prerequisite remote learning in advance of in‐person patient care activities can deliver clinical education with similar student evaluations and knowledge test performance to a traditional model. Beyond times of crisis, such a model may help institutions to address other challenges of a medical school's clinical curriculum.

CONFLICT OF INTEREST

The authors have no conflict of interest to disclose.

ETHICAL APPROVAL

All student data were obtained from Case Western Reserve University School of Medicine's Medical Education Data Registry whose purpose is to facilitate education research and quality improvement. The IRB protocol (IRB202151105) was a de‐identified data registry that was determined as exempt by the institution. As it was exempt, signed consent was not necessary from each student.