Lessons From Learners: Adapting Medical Student Education During and Post COVID-19.

In response to the COVID-19 pandemic, many medical schools suspended clinical clerkships and implemented newly adapted curricula to facilitate continued educational progress. While the implementation of these new curricula has been described, an understanding of the impact on student learning outcomes is lacking. In 2020, the authors followed Kern's 6-step approach to curricular development to create and evaluate a novel COVID-19 curriculum for medical students at the University of California San Francisco School of Medicine and evaluate its learning outcomes. The primary goal of the curriculum was to provide third- and fourth-year medical students an opportunity for workplace learning in the absence of clinical clerkships, specifically for students to develop clerkship-level milestones in the competency domains of practice-based learning and improvement, professionalism, and systems-based practice. The curriculum was designed to match students with faculty-mentored projects occurring primarily in virtual formats. A total of 126 students enrolled in the curriculum and completed a survey about their learning outcomes (100% response rate). Of 35 possible clerkship-level milestones, there were 12 milestones for which over half of students reported development in competency domains including practice-based learning and improvement, professionalism, and interpersonal and communication skills. Thematic analysis of students' qualitative survey responses demonstrated 2 central motivations for participating in the curriculum: identity as physicians-in-training and patient engagement. Six central learning areas were developed during the curriculum: interprofessional teamwork, community resources, technology in medicine, skill-building, quality improvement, and specialty-specific learning. This analysis demonstrates that students can develop competencies and achieve rich workplace learning through project-based experiential learning, even in virtual clinical workplaces. Furthermore, knowledge of community resources, technology in medicine, and quality improvement was developed through the curriculum more readily than in traditional clerkships. These could be considered as integral learning objectives in future curricular design.

The COVID-19 pandemic drastically and abruptly changed the global medical education landscape. Most medical schools were forced to alter planned curricula and suspend clinical clerkships, necessitating innovation to continue training students. [1,2] As medical educators move from the initial phase of immediate curricular adaptations due to COVID-19 into the next phase of medical education design, there is an urgent need to examine outcomes data to articulate learning gaps and guide future innovations.

In the face of limited to no availability of clinical clerkships at the outset of the COVID-19 pandemic, undergraduate medical education leaders redesigned workplace learning experiences for clerkship students. [3] Workplace learning is experiential learning in the clinical setting through which learners apply the clinical knowledge, skills, and behaviors of their professional roles and receive feedback in real time. [4] Clerkships are vital for medical student workplace learning, [5-7] enabling participation in clinical activities through workplace affordances that allow exploration of tasks and activities, relationships and interpersonal dynamics, and rules and norms. Workplace learning encourages learner engagement and shapes knowledge, skills, attitudes, and beliefs. [8,9]

To offer clinical learning opportunities without extending time to graduation, educators created novel workplace learning experiences for clerkship students. These new student roles mitigated exposure risk and involved students interfacing with patients via telemedicine, coordinating public health responses, and developing patient, provider, and community resources. [10,11] While the implementation of some of these COVID-19-adapted medical curricula has been previously described, an understanding of student learning outcomes is, to our knowledge, lacking. [12-15]

We therefore set out to apply Kern’s 6-step approach to curriculum development to the design and evaluation of a novel COVID-19 curriculum for clerkship medical students. [16-19] Those 6 steps are:

Problem identification and general needs assessment;

Targeted needs assessment;

Goals and objectives;

Educational strategies;

Implementation; and

Evaluation and feedback.

We analyzed student projects, motivations, competencies, and learning themes. Here, we report medical student learning outcomes and lessons learned to inform education innovations aimed at engaging and empowering students in workplace learning during and after the COVID-19 pandemic.

Application of Kern’s Framework to the COVID-19 Response Curriculum

Step 1: Problem identification and general needs assessment

In March 2020, the Association of American Medical Colleges recommended pausing clinical clerkships because of changed circumstances due to COVID-19. [20] That week, clinical clerkships were suspended at the University of California San Francisco School of Medicine (USCF), leaving third- and fourth-year medical students lacking a mechanism for workplace learning experiences.

Step 2: Targeted needs assessment

The associate dean for curriculum charged a group of faculty involved in project-based curriculum development for preclinical students with creating an educational opportunity for clerkship-level students. Four clerkship medical students participated in virtual focus groups with this curricular team and identified 2 priority needs: learners’ need for safe and experiential workplace learning and the health system’s need for additional support responding to new and changing problems due to conditions created by COVID-19.

Step 3: Goals and objectives

The primary goal of the UCSF COVID-19 response curriculum was to provide an opportunity for workplace learning in the absence of traditional clinical clerkships. Specifically, our aim was for students to fill identified health systems gaps, and in doing so, develop clerkship-level milestones in the competency domains of practice-based learning and improvement, professionalism, and systems-based practice. Development in the competency domains of patient care, medical knowledge, interpersonal and communication skills, and interprofessional collaboration were not primary goals because student participation in direct in-person patient care was discouraged and students would likely be working primarily via virtual platforms. An additional goal was to enable students to continue career and specialty exploration.

Specific course objectives were to:

Describe the challenges associated with altered health system resources and restrictions on traditional physician–patient interactions in a pandemic;

Articulate the changing roles of the health care team and the medical system to continue to provide excellent patient care in the pandemic environment;

Within the context of the health care team, seek and select evidence-based strategies for an identified gap around clinical quality, safety, patient experience, or value;

Contribute to care delivery efforts to improve the quality, safety, patient experience, or value for a panel or population of patients during a pandemic;

Respond to mesosystem (e.g., community or public health) needs regarding access to care, health and health care disparities, and intersystem links between care providers (e.g., hospital to clinic, prison to community); and

List one’s own roles and responsibilities and those of professionals on the health care team and seek feedback throughout the experience.

Step 4: Educational strategies

The COVID-19 response curriculum was designed to match clerkship medical students with projects led by faculty at 3 UCSF-affiliated health systems: the UCSF Health system, the city and county San Francisco General Hospital (ZSFG)/Department of Public Health, and the San Francisco Veterans Affairs (SFVA) Health Care System.

Faculty-mentored COVID-19 clinical projects for students were identified by health system site teams, composed of faculty and students affiliated with each site. These teams identified faculty partners from a variety of specialties to solicit and brainstorm experiences that would target the pandemic response. Faculty were encouraged to design experiences that could accommodate individual student availability, such that students could determine their own weekly participation schedules. Site teams anticipated that this structural flexibility could empower students to take ownership of their learning in the rapidly evolving context of the pandemic.

Faculty with potential projects met with site teams to discuss logistics and feasibility before submitting a project proposal. Site teams provided guidance to project leaders and assessed the educational value of the opportunities. Projects that did not meet logistical requirements (e.g., required exposure to COVID-positive patients, conflicted with mandatory student didactics) or were assessed as not providing sufficient educational value (e.g., lacked workplace learning component, data-entry only roles) were not assigned students.

Step 5: Implementation

All third- and fourth-year students were offered the opportunity to sign up for the curriculum and none were required to enroll. Students were queried by electronic survey about their health system onboarding access, scheduling availability, language fluency, and general interests. Students were then matched to projects by the curricular design team, with special consideration given to matching students to projects relating to their interests (e.g., pediatrics, population health). The number of students matched to each project was based on project need. Elective units, not required for graduation but reflected on students’ final transcripts, were offered for participation. Students received a pass/no pass grade for this curriculum.

We encountered several challenges while establishing the curriculum. First, the immediate high demand from clinical sites interested in student participation challenged our administrative capacity. This high demand was likely due to the increased burden on the health system from COVID-19 combined with our clinical faculty’s enthusiasm to continue teaching students. We addressed this difficulty by triaging projects by urgency, quickly matching students to active projects and efficiently supporting faculty with timely and short planning meetings. Second, we were challenged to ensure equity for students when assigning projects, as some students were interested in participating in multiple projects and had experience and skills applicable to several projects. We offered all students 1 project before allowing students to participate in multiple projects. Due to the high degree of faculty engagement, we were able to offer sufficient and equal opportunity for students to participate in projects aligned with their interests.

Step 6: Evaluating the effectiveness of the curriculum

We queried students by electronic Qualtrics survey following completion of the curriculum (version XM; Qualtrics, Provo, Utah; 2020) (the full survey may be found at https://ucsf.co1.qualtrics.com/jfe/form/SV_cZvkmbVNOov984B). The survey explored students’ narrative motivations, goals, and learning outcomes and also included a checklist for learners to self-identify UCSF clerkship-level competency milestones developed during the curriculum in competency domains of patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, systems-based practice, and interprofessional collaboration. [21] Students were required to complete this survey to receive academic credit for curriculum participation.

Survey responses were de-identified before coding. We tabulated quantitative data, including student appraisal of clerkship-level competency milestones developed during participation in the curriculum, in Excel (version 16.41; Microsoft; 2020).

Qualitative data included student responses about motivations for participating in the curriculum and learning points taken away from the curriculum. We conducted thematic analysis using the 6-step method described by Kiger and colleagues, adopted from Braun and Clarke’s model. [22,23] An inductive approach was taken, deriving themes from the data without a deductive preexisting framework. First, 3 authors involved with development of the curriculum, namely, 2 third-year medical students (L.M.C. and M.R.H.C.) and 1 faculty member (S.E.F.), familiarized themselves with the data by individually reading through all student survey responses. Then, responses were divided and distributed to each of the 3 authors, who began generating initial codes within their subset, allowing for multiple codes per response and reviewing responses as new initial codes emerged. The team then compared, contrasted, and combined initial codes identified within their subset of responses. Using these updated codes, the team synthesized and mapped out themes that emerged. After developing this set of themes and subthemes, authors (L.M.C., M.R.H.C., and S.E.F.) reviewed it to ensure these accurately reflected responses. Team members were then assigned a different subset of responses to review to ensure correct assignment and accurate reflection of themes and subthemes, such that all responses were reviewed by at least 2 team members. Before data analysis, the student authors received training in thematic analysis from faculty authors with extensive experience in this method.

Student Learning Outcomes

Student participation and project characteristics

A total of 126 medical students enrolled in the curriculum and submitted their postcurriculum survey (100% response rate). Student demographics are presented in Table 1. A total of 38 projects across the 3 health systems (academic, county, and veterans) were created, of which 32 (84%) were completed remotely. Distribution of projects across departments was as follows:

Table 1

Demographics of COVID-19 Response Curriculum Participants, University of California San Francisco School of Medicine, 2020

Internal medicine: 20 (53%)

Obstetrics–gynecology: 4 (11%)

Public health: 3 (8%)

Emergency medicine: 3 (8%)

Pediatrics: 3 (8%)

Family and community medicine: 2 (5%)

Interventional radiology: 2 (5%)

Psychiatry: 1 (3%)

Projects addressed a range of health system gaps, spanning 8 specialties and a variety of patient populations. Projects’ aims included providing inpatient or outpatient support, assisting public health efforts, creating patient and community or provider resources, and engaging in telemedicine (see Table 2).

Table 2

University of California San Francisco School of Medicine COVID-19 Response Curriculum Project Topics and Departments

All interested students were matched with 1 or more projects. The number of students assigned to each project ranged from 1 to 11. Mean project duration was 7 weeks (range, 1–20 weeks). Sixty (48%) students participated for less than 10 hours per week, 48 (38%) for 10 to 20 hours per week, 9 (7%) for 20 to 30 hours per week, 3 (2%) for 30 to 40 hours per week, and 6 (5%) for more than 40 hours per week. All students met the objectives of the curriculum and received a passing grade for participation.

Competency milestones

Student-identified clerkship-level competency milestones achieved through participation in their COVID-19 curricular project are presented in Table 3. [21] Of the 35 possible milestones, there were 12 milestones for which the majority of students reported development: practice-based learning and improvement milestone 2 (57%); interpersonal and communication skills milestones 1 (86%) and 4 (60%); and professionalism milestones 1 (73%), 2 (87%), 3 (64%), 4 (76%), 5 (80%), 6 (67%), 7 (68%), 8 (56%), and 9 (66%). The most commonly identified competency domains for which over half of students reported development of milestones were professionalism (all milestones), interpersonal and communication skills (half of milestones), and practice-based learning and improvement (one-third of milestones).

Table 3

Clerkship-Year MD Competency Milestones Explored by 126 Students Participating in the COVID-19 Response Curriculum, University of California San Francisco School of Medicine, 2020

Student motivation

Students described 2 central motivations for participating in the curriculum: fostering their identity as physicians-in-training and patient engagement.

Identity as physicians-in-training.

The most prominent motivation mentioned by participants was a “desire to contribute,” arising from a sense of responsibility tied to their identity as physicians-in-training. Students strongly identified as a part of the health care system, feeling simultaneously called to action but lacking opportunities to participate and develop as clinicians. Recognizing the increased demands and decreased support clinical health care workers were facing, many learners were motivated to alleviate health care worker burden. Students also described a sense of responsibility to use specific skill sets, including language abilities, laboratory training, or proficiency from employment before medical school. Sample learner quotes include:

I felt compelled to contribute to pandemic response as a member of the larger medical profession. I was saddened by our removal from the clinical environment and was searching for another mechanism by which to help the health system. (Third-year student, Department of Medicine, UCSF Health)

I felt pretty trapped when we were pulled from clinical rotations…. It was hard to reconcile this desire to help and put forth my best effort to aid in the effort to provide thorough, equitable, and quality care to people in need. (Third-year student, Department of Medicine, SFVA)

I wanted to use my time to be helpful, especially when it’s clear that many physicians, nurses, and other health care workers are sacrificing so much. (Third-year student, Department of Psychiatry, ZSFG)

Patient engagement.

Students were eager to engage with patients to maintain connection and mitigate isolation for both patients and students. Specifically, some participants recognized how social distancing and hospital visitor restrictions could detrimentally affect patients’ emotional wellbeing. Learners were also motivated to join the curriculum to address patient health needs, as they worried about the many burdens felt by patients, such as limited health care access and the substantial health threat of COVID-19. Others joined out of concern about the disproportionate impact of the pandemic on vulnerable patients, including older adults, racial minorities, women, and those with limited English proficiency or lower economic means. Sample student quotes include:

With COVID-19 restrictions, families and loved ones have been separated from patients at a time when they are most in need of love and support. I was primarily drawn to this [curriculum] because of its tangible impact on patient experience and patient care during such difficult times. (Third-year student, Department of Medicine, UCSF Health)

I wanted to be able to address disparities in the COVID-19 pandemic. (Third-year student, Department of Medicine, ZSFG)

I want to care for people when they are most vulnerable and support them through the hardest times. (Third-year student, Department of Medicine, SFVA)

Student learning

Students’ qualitative survey responses conveyed 6 central learning themes: interprofessional teamwork, community resources, technology in medicine, skill-building, quality improvement, and specialty-specific learning (Table 4).

Table 4

Sample Quotations for Student Learning Themes Developed During the COVID-19 Response Curriculum

Interprofessional teamwork.

Medical students described learning from interprofessional team members, including social workers, laboratory technicians, nurses, and pharmacists. Specifically, they witnessed how the roles of each team member are coordinated to produce an efficient team, maximizing patient health. The importance of collaboration and communication within and across teams was impressed upon learners who recounted how teamwork had been successful or could have been improved. Learners also engaged with the clinical support processes that enable the clinical workplace to function. They detailed a newfound understanding of the complexities and efficiencies, or lack thereof, of many health systems processes, including managing orders, lab processing, and consulting other services.

Community resources.

Many participants reported learning about the workflow, processes, and impact of community resources on patient care. Students described expanding their knowledge about the services provided by community programs, government programs, and not-for-profit organizations, noting how these services are necessary for adequate and equitable health care and help to address disparities exacerbated by the pandemic. Learners also commented on how complicated accessing these services can be and how little training medical school provides regarding this process. Further, many responses detailed a new understanding of skills needed for delivering medical care in the community, including establishing connections, building institutional trust, and developing targeted programs to support specific communities.

Technology in medicine.

Students described participating in various health care technology initiatives and gaining an appreciation for how telehealth can meet patient needs. They articulated how telehealth allowed patients who might otherwise have been unable to access care, due to burden from travel, work, and/or childcare, to quickly and relatively easily connect with their health care provider. Learners also chronicled how technology such as virtual platforms could be employed to combat patient isolation, including connecting hospitalized patients unable to have visitors with their family members and providing social connection to lonely elderly community members.

Skill-building.

In their new and varying roles, participants sharpened their patient communication and public health skills. Specifically, they practiced difficult communication when they delivered bad news, such as positive COVID-19 test results, to patients. They applied complex communication skills to obtain the best possible medication history, provide contraception counseling, and conduct medical histories and physical examinations by telehealth. Learners further developed their public health skills by participating in contact tracing, translating health materials for the public, and working with epidemiologists. Students also articulated the profound impact empathy can have on patients’ emotional wellbeing and learned that any member of the clinical team can offer this substantial benefit to patients. By genuinely listening and providing compassionate support, they learned to develop a therapeutic alliance with patients.

Quality improvement.

Learners engaged in quality improvement at the level of teams, departments, and systems as they conducted Plan-Do-Study-Act cycles, [24] continually reviewing and updating their projects based on feedback and outcomes. Students were enthusiastic and encouraged by this fast-paced, iterative process and expressed interest in participating in quality improvement work in their future professional endeavors.

Specialty-specific learning.

Participants explained how specific projects enabled them to explore their specialty interests, including in pediatric ophthalmology, radiology, obstetrics–gynecology, and emergency medicine. This filled a gap in students’ available opportunities to explore their career interests and allowed them to develop relationships with faculty in specific specialties.

Lessons Learned

While clerkships are widely recognized as the central mechanism for clinical learning, [5-7] students can have rich workplace learning experiences in nontraditional workplaces. Traditionally, clinical skills and competencies are learned through an apprenticeship model in direct patient care, but the curriculum we developed demonstrates how virtual clinical workplaces and project-based teaching strategies can also enable clerkship students to develop competencies and achieve valuable workplace learning. This is not to suggest that clinical clerkships can successfully be conducted entirely remotely or in a project-based format, but instead that traditional clinical clerkships could be extended to include other learning opportunities and clinical settings (e.g., quality improvement, telemedicine, public health). For example, with the return to traditional clerkships, students could continue to be involved in telehealth outreach to vulnerable patient populations to address their health needs.

We found that an advantage of this curricular structure was the flexibility afforded to students to guide their own learning. Students primarily set their own schedules and often conducted projects remotely. Almost half of the projects required less than 10 hours per week. Many medical schools have recently focused on adapting curricular structure to support student wellness, [25] and as medical schools move curricula to virtual settings, we caution against overloading students with rigid, lengthy virtual learning and inducing virtual platform fatigue. [26] Our findings indicate that students can achieve important learning when they are empowered with flexibility in their education and schedule and when the learning opportunities match their interests and serve the community.

This curriculum enabled students to learn important clinical doctoring skills and health systems science knowledge that are missing from clinical clerkships as they are currently designed. [27,28] Some of the themes students’ learning embodied, such as interprofessional teamwork and clinical skill-building, are typically included in undergraduate medical education. However, familiarity with community resources, technology in medicine, and quality improvement, described by students and recognized by educators as essential components of medical education in preparing students to be effective physicians, have been less consistently included as learning objectives in the traditional clerkship curriculum. [11,29-31] We posit that a curricular model such as the one we describe here could be incorporated to complement traditional clerkship experiences as a means of including these important health systems science topics throughout undergraduate medical education. [27,28]

Observations and Conclusions

Our application of Kern’s framework to a novel COVID-19 curriculum sheds light on the student experience of COVID-19 curricula and may guide future curricular innovations in the context of a sudden and massive global or national event affecting medicine and medical education.

Our COVID-19-adapted curriculum achieved our primary goal of providing students with opportunities to engage in clinical workplace learning to advance the medical school’s educational competencies. A majority of students reported development in clerkship-level competency milestones in our target competency domains of professionalism and practice-based learning and improvement, and surprisingly in interpersonal and communications skills as well. [21]

Our thematic analysis of individual student experiences demonstrated that students cultivated important skills, including interprofessional teamwork, knowledge of community resources, technology in medicine, quality improvement, skill-building, and specialty-specific learning. Specialty-specific learning matched to career interests was a difficult aim to achieve during the pandemic otherwise but was possible with this curriculum.

Strengths of our program evaluation include our intentional design of the evaluation strategy at the start of the curriculum and our systematic methods of thematic analysis. Multiple researchers reviewed each survey response and iteratively discussed and reevaluated the themes and subthemes we developed. A limitation of our evaluation is the lack of additional assessments to supplement data on students’ self-reported learning. However, as students received a passing grade for participating in the curriculum, there was no incentive for students to overreport learning that did not occur.

As we transition back to prior curricula or continue with newly created educational formats, we must heed these lessons and intentionally design educational experiences that maximize student growth and prepare students to be effective physicians. This includes creatively providing workplace learning opportunities, integrating topics central to health care delivery, such as community resources, technology in medicine, and quality improvement, and empowering students with flexibility in their learning environment.

Acknowledgments:

The authors acknowledge the commitment and creativity of the faculty and students who participated in the University of California San Francisco School of Medicine COVID-19 response curriculum. The authors thank Therese Denoga and Patricia Runquist for their assistance with curriculum implementation and data collection, as well as Dr. Bridget O’Brien for her manuscript review and thoughtful guidance.