Pivoting to a Remote-Learning Summer Student Program during the COVID-19 Pandemic.

Research experience garnered through summer student programs (SSPs) is critical for high school and college student retention in science, technology, engineering, and math (STEM) disciplines. However, the global coronavirus disease (COVID-19) pandemic prevented in-person SSPs in 2020, eliminating these essential experiences that students need to advance their STEM training. In response, we created a remote-learning model that can be broadly adapted for other SSPs. We aimed to uphold our traditional SSP's academic rigor by cultivating critical thinking skills, providing mentorship, and equipping students with tools to serve as public health ambassadors in their communities. We designed the remote SSP around an anchor topic to integrate didactic lectures with research-based independent projects. Program success was evaluated quantitatively and qualitatively via content assessments and written feedback. By comparing preassessments to postassessments, we show that students gained general scientific literacy and improved critical thinking skills. Based on qualitative measures, students were satisfied with their mentorship, reported that they would use what they learned through the SSP in the future, indicated that they had the tools to understand and communicate public health information, and, overall, rated the quality of the SSP positively. As the pandemic continues to necessitate remote learning, traditional in-person experiences will need to be adapted to best support students. We have developed a modular and adaptable SSP that upholds the same standards as the traditional SSP by continuing to provide essential experiences necessary to advance students' training in STEM.

There is a projected shortage of science, technology, engineering, and math (STEM) professionals in the United States (1). Concomitantly, students’ pursuit of STEM-focused careers is declining, in part because of misperceptions of the STEM field and the lack of resources to participate in STEM programs (2–4). Summer student programs (SSPs) are critical for improving high school and college students’ retention in STEM (5–8). Research experience gained through SSPs is crucial for students’ retention in science majors and can serve as a steppingstone to careers in STEM, particularly for students who identify as underrepresented minorities (URM, defined here as Black or African American, Hispanic or Latino, American Indians or Alaska Natives, Native Hawaiians, and other Pacific Islanders) and/or women (9, 10). Additionally, participation in research has been recognized as one of the fundamental approaches for maintaining student interest in STEM disciplines (11).

The Division of Pulmonary and Critical Care Medicine at Northwestern University has been hosting an SSP for more than two decades. The program is dedicated to providing enriching research training to high school and university students interested in the biological sciences, with priority given to women and URM students. The backbone of in-person SSPs was a dedicated research experience: students were paired with a mentor and participated in hands-on, wet-bench research for a period of 8–12 weeks. During the in-person SSP, we emphasized three goals: to cultivate critical thinking skills, to provide mentorship, and to train students to be public health ambassadors in their communities.

Unfortunately, the coronavirus disease (COVID-19) pandemic has halted in-person research activities, including traditional SSPs. To remedy this limitation, many institutions, including ours, have been forced to find alternative ways to support students interested in pursuing STEM (12). Because of the continued urgency to provide summer research experiences for high school and college students, we created an initiative with which to carry on the SSP remotely while still retaining the three key tenets of the in-person SSP. We outline the differences between in-person and remote-learning SSP in Table 1. In this manuscript, we report how we transitioned the SSP from an in-person to a remote-learning program and our evaluation of the effectiveness of the online SSP. We describe our highly adaptable learning model as a resource for other academic institutions’ SSPs. We show that, amid a global pandemic, our program could continue supporting STEM education without compromising the quality of the learning and research.

Table 1.

Differences between in-person learning and remote-learning summer student program

	In-Person Learning	Remote Learning
Program length	8–12 wk	4 wk
Mentor to student ratio	1 to 1	1 to 2.5
Core lectures	No	Yes
Student journal club	Yes	Yes
Lunch and learn	Yes	Yes
Professional development	No	Yes
Clinical observation	Yes	No
Guided discussion	No	Yes
Stipend provided	Yes	Yes
Program evaluation	End of the program	Weekly
Pre/post assessment	No	Yes
Research/learning theme	Varied	COVID-19/viral pneumonia
Research project	Individual	Small group of 4–5
Final research deliverables	Poster/oral presentation	Live online presentation

The Transition of In-Person to Remote-Learning Curriculum via an Anchor Topic

We received more than 60 applications from high school and college students across six states. The program enrolled 21 students (14 high school juniors and seniors and 7 undergraduate students). Forty-five percent of the accepted students admitted to our SSP identified as URM, and 68% identified as female. To continue offering hypothesis-driven research and learning experiences to the students, we divided the curriculum into Block A (9:00 a.m. to 12:30 p.m.) and Block B (1:30 p.m. to 4:00 p.m.) learning blocks, which were anchored around the topic of COVID-19. We chose COVID-19 because of its relevance to both our student population and our division at large, and also because it served as a strong common platform for integrating different areas of knowledge (basic biology, immunology, epidemiology, and research techniques) and skills (critical thinking, problem-solving, teamwork, and presentations) for cultivating students’ interest in STEM and for training students to become public health ambassadors. Block A consisted of synchronous lectures and guided discussions. Faculty, postdoctoral and clinical fellows, graduate students, and research technicians led these sessions on the topics listed in Table 2. Leveraging support from trainees across our department was a central tenet of the SSP; this approach both helped to distribute labor equitably across laboratories and provided an opportunity for trainees to gain valuable mentorship experience. In addition, tutorial videos were assigned to complement lecture material each week. This form of asynchronous learning helped reinforce concepts in the lectures and also helped reduce “Zoom fatigue” brought on by the morning lectures. Gaps in student knowledge, after attending the synchronous lectures and viewing the tutorial videos, were addressed using problem-based learning strategies via discussions led by either a clinical fellow or medical student.

Table 2.

Overview of Block A and Block B

Week	1	2	3	4
Block A (morning sessions)
Lecture topics	Introduction to the innate immune responsePCR and antibody testing for COVID-19Critically thinking in experimental design	Introduction to the adaptive immune responseImmune response in viral infectionIntroduction to RNA-sequencing and bioinformaticsEpithelial cell signaling	Current findings in COVID-19Drug treatment for COVID-19	Introduction to epidemiologyFinal student project presentations
Research techniques	Quantitative PCR, ELISA	Flow cytometry, RNA-sequencing	Immunohistochemistry, statistical analysis	Clinical Research
Journal club	Susceptibility of ferrets, cats, dogs, and other domesticated animals to SARS-CoV-2 (15)	Complex Immune Dysregulation in COVID-19 Patients with Severe Respiratory Failure (16)	A Randomized Trial of Hydroxychloroquine as Post-exposure Prophylaxis for COVID-19 (17)	Modeling the COVID-19 epidemic and implementation of population-wide interventions in Italy (18)
Additional learning resources	How is coronavirus diagnosed? (19)How to Use PubMed (by Northwestern Galter Library)	Epidemiology and pathophysiology of COVID-19 (20)	Introduction to Statistical Analysis (21, 22)Coronavirus treatment, prognosis, and precautions (23)	Patient sample collection (24)Donning and doffing PPE (25)Performing bronchoscopy (26)
Professional development activities	Lunch and Learn: Applying to Graduate SchoolResearch Confidence and IndependenceEquity and Inclusion Awareness Skills	Lunch and Learn: Applying to Medical SchoolCommunicating Research FindingsComposing a Competitive Personal Statement	Lunch and Learn: Applying to CollegeThe Graduate School Application Plan	End-of-program evaluation
Block B (afternoon sessions)
Weekly research goals	Background researchBrainstorm research question(s)Form hypothesis	Research approaches/methodsAdditional research tools	Data collectionData analyses	Prepare research presentationsPresent research projectFinal evaluation

Definition of abbreviations: COVID-19 = coronavirus disease; ELISA = enzyme-linked immunosorbent assay; PCR = polymerase chain reaction; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2.

We designed Block A of the SSP to prime students with the basic scientific information needed to understand the pathogenesis of COVID-19 and to perform research projects in Block B (Table 2). This structure, generally speaking, moved from broad to specific. We began with the basics of viral infections and the host response before moving on to the pathophysiology of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection, patient-specific treatment options, and COVID-19 epidemiological trends. New knowledge was built gradually from previously weeks, adding more complexity and specialty as we moved along the program. The more intensive Block A lectures were concentrated in Weeks 1 and 2 of the program as students built a foundational knowledge of the material and were pared down in Weeks 3 and 4 as students shifted their focus to their Block B independent group projects.

The SSP Improved Understanding of COVID-19 and Strengthened Critical Thinking Skills

The goal of Block B was to help students develop critical thinking, teamwork, and presentation skills. Groups of 4–5 students worked together on original group research proposals with mentors from different laboratories during the asynchronous Block B. Group mentors, who were mainly faculty and postdoctoral fellows, had full autonomy over students’ research proposals and progress. Their role was to provide guidance and feedback to help students achieve the weekly research goals set by the SSP (Table 2). These goals included identifying a research question, using PubMed to gather relevant background information, identifying appropriate experimental and analysis methods, and preparing a final presentation. Students worked in groups without their mentors but were required to check in daily and to have more in-depth meetings as their mentors saw fit. We collected weekly feedback to ensure the students were on track. The projects focused on specific areas of concentration related to COVID-19, including cytokine storm, the immune landscape, imaging techniques, targeted drug treatment, and neurological symptoms of the disease. Students proposed research questions related to their assigned COVID-19 topics and designed a series of experiments to test their hypotheses, using knowledge from core lectures and research techniques they learned in Block A. For example, one group of students applied the knowledge learned from core lectures on immune responses to viral infection, enzyme-linked immunosorbent assay, and flow cytometry techniques and integrated what they learned using the PubMed search engine to propose a study on cytokine profiling of patients with COVID-19 in their research project. At the end of the program, each group of students presented a project proposal in line with their given topic and received oral feedback from faculty and trainees within the Pulmonary Division.

To quantitively assess critical thinking skills gained through the program, students were subjected to a pre- and postassessment. Questions were designed to test general knowledge about COVID-19 and to measure how well students could integrate knowledge to answer more complex questions (Appendix 1 in the data supplement). Students displayed mastery of biologic concepts related to the COVID-19 pathogenesis (Figure 1, Q4, Q7, Q12) and COVID-19–specific information (Figure 1, Q1–3, Q6, Q11, Q13). Students also displayed improvement on all questions that tested their critical thinking skills (Figure 1, Q5, Q8–10, Q15). For example, most students correctly chose the best way to experimentally test for proteins (Figure 1, Q8). Overall, the percentage of correct responses was significantly increased for all of the questions during postassessments. Of note, one student wrote in their exit survey, “Although I didn’t get a chance to continue with hands-on laboratory work, I feel like I learned so much more about critical thinking and how a scientist thinks when designing their experiments, which is just as important and arguably more than just learning experimental techniques themselves.” Together, these results indicate that students retained content and improved their critical thinking skills over the course of the SSP.

Figure 1.

Pre- versus postassessment evaluations of summer students on knowledge related to coronavirus disease (COVID-19). A total of 15 assessment questions were used to evaluate students’ general knowledge on cell biology, research techniques, and COVID-19 on the first and last days of the summer student program. Percentage of correct responses between pre- and postassessment are compared in bar graphs (right panel), corresponding to questions 1–15 listed on the left. Complete assessment questions are listed in Appendix 1 in the data supplement. SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2.

Fostering an Online Community of Mentors to Promote Learning Objectives

Throughout the SSP, mentorship took several forms. First, small groups of students were directly matched to a division member who served as their primary research mentor for Block B projects. The mentors’ role was to guide students through a process of experiential learning (13). Through this approach, students learn through a cycle of hands-on experience and reflection. Specifically, this approach takes place in four phases: concrete experience, reflective observation, abstract conceptualization, and active experimentation. Students first garnered “concrete experience” through didactic lectures in Block A. This information was reinforced with “reflective observation” through discussions, weekly feedback solicitation, and informal conversations with mentors and peers. In Block B, students experienced the “abstract conceptualization” phase of experiential learning by applying what they were taught in Block A and to address their assigned scientific topics. This allowed students to directly test their understanding of underlying principles, processes and procedures obtained in Block A. Finally, students took part in “active experimentation”; they synthesized information and skills they developed from both content blocks to present to one another in discussion sessions and to ultimately create a project proposal that they presented to the entire department. Working through each of these four tenets of experiential learning created a feedback loop, whereby students took part in problem-solving exercises, received feedback from their mentors and peers, and then continued attempting to solve problems. Because of the compact nature of the material, students received rapid feedback and could immediately identify areas for improvement.

This approach was validated by 100% of participants who either strongly agreed or agreed with the statement, “my mentors are approachable, and I feel comfortable asking questions/discussing matters with them” (Figure 2A). Moreover, 73% of participants strongly agreed with the statement, “I have continued to receive constructive and timely feedback from my mentors on my work” (Figure 2B). Additionally, students received mentorship through a series of lectures related to career planning. For example, we hosted a weekly “Lunch and Learn” series, in which we enlisted trainees to participate in an informal question-and-answer session covering topics such as college, graduate school, and medical school admissions. Our program also offered professional development opportunities and learning resources by partnering with The Graduate School at Northwestern University in hosting workshops on writing personal essays, developing independent research projects, and communicating research findings (Table 2). These opportunities, together with Block B research experience, helped students clarify their career goals, with one student reporting, “I definitely want to dive deeper into [computational projects] in the future.” Interestingly, students also reported experiencing peer-to-peer mentorship. Overall, 80% of participants either agreed or strongly agreed with the statement, “I prefer smaller group discussions to big group discussions” (Figure 2C). One student wrote, “I think working in small groups for the project was very effective for building collaboration skills and allowed us to learn from each other while supporting each other.” This feedback indicates that students typically preferred small-group, learner-centered approaches. When asked to rate the quality of the professional network built through the program, 53% of participants rated the quality as “good” or “excellent” (Figure 2D). These reports suggest that students received valuable mentorship from a diverse group of people. Building an effective professional network is an important goal of the SSP. However, because of the nature of the remote-learning SSP, the opportunity for in-person interactions between peers and mentors was eliminated. Although students were given many opportunities to network with their peers, mentors, and speakers remotely, some of them did not feel that the networks that they built were as effective as the ones they built in person. This was especially true for professional networking events that we hosted in collaboration with The Graduate School on career development. This was partially because of the lack of structured networking outlets and dedicated networking time after the events. We also observed that students were hesitant to ask questions or speak up in networking events with large audiences. These factors likely contributed to the lower rating of the professional network quality.

Figure 2.

Summer student program end-of-program evaluation. Shown are selected questions and responses from the end-of-program evaluation. Overall, students rated the summer student program very positively.

SSP Trains Participants as Public Health Ambassadors

Misconceptions surrounding COVID-19 are common (14). To train students to disseminate science-based knowledge of COVID-19, we offered guided discussion sessions as part of the reflective observation stage of experiential learning. First, a guided journal club discussion was designed for students to critically analyze breakthroughs on COVID-19. Students were assigned to a scientific article each week (Table 2). Groups of 3–4 students each presented one figure and posed a discussion question for the rest of the group. Students also participated in a guided news discussion. In small groups, they prepared comparisons of different news stories that covered the same COVID-19–related breakthrough. Students related these media reports to their original scientific data and evaluated whether the media’s portrayal accurately recounted the scientific findings. These guided discussion formats helped train students in using science-based knowledge to ensure the spread of accurate information.

When asked if the SSP changed students’ perceptions of COVID-19, 89.5% answered that it had. One student replied, “I would say this program definitely gave me an amazing education, not just about COVID-19, but about how to assess the news and trials featured in the media critically.” Other students echoed the idea that students felt better equipped to analyze media coverage of COVID-19–related topics. Students indicated that they were going to take the pandemic more seriously following the completion of the program. For example, one student said, “[The SSP] taught me that COVID-19 is not a disease you should take lightly and that even if I am healthy, I can still infect and harm other people. While before, I used to push to go to school for this year, now I understand why it is so dangerous.” Students also discussed using the knowledge they gained in the program to inform their peers, “I have friends who easily buy into a thing they hear, and I have to tell them that they shouldn’t go running toward an idea according to evidence 1, 2, and 3.” Another student wrote, “This program has helped me to explain to members of my family and community scientific information that is true to combat the massive amounts of false information.” Taken together, these results suggest that students developed the essential skills required to evaluate COVID-19–related media reports and to serve as ambassadors of science-based knowledge within their communities.

Evaluating the Overall Success of the Program

Students were asked, “How much do you think the materials you learned in this summer program will have practical applications in your life outside of this program?” In response, 68% of the students answered, “Very much” and 21% answered “To the greatest extent” (Figure 2E). Specifically, one student cited “The Prism graphing, searching PubMed, and just how to organize a slideshow for a presentation” as skills they would apply in the future. Students also noted collaboration, critical thinking, and experimental design as skills they built throughout the program. Overall, 100% of the students rated the quality of the program as either “Excellent” or “Good,” suggesting that they considered their time in the SSP a positive experience (Figure 2F). Together, these results indicate that students were enriched by their SSP experience and plan to use what they learned in future endeavors.

Conclusions

This manuscript presents an effective model for transitioning from an in-person to a remote-learning model SSP. Specifically, we met the same goals as our traditional SSP, including cultivating critical thinking skills, providing mentorship, and equipping students with tools to serve as public health ambassadors. We showed, through qualitative and quantitative metrics, that the remote-learning SSP improved scientific literacy and critical thinking skills, provided quality mentorship, and prepared students with the tools to understand and communicate public health information. Overall, students rated the quality of the program highly. Although it is likely that our students will return to in-person research this summer, this highly adaptable remote-learning model can be feasibly modified for dynamic hybrid learning in the future to complement the in-person learning model of SSP with any other themes and topics. For example, core lectures can be prerecorded and/or take place remotely to provide both mentors and students with more flexibility and accessibility. Online platforms, such as Zoom and Slack, can also provide an additional space for students to work collaboratively outside the laboratory setting. Incorporating elements of remote learning into our traditional SSP will improve accessibility for students who are not able to be in the laboratory 40 hours a week. Together, these modifications will provide a more robust learning experience for all students.

As we continue to build on this model, we note several challenges that we would address in future iterations of this program. First, students reported experiencing “Zoom fatigue” in their qualitative assessments. Therefore, we recommend shortening lecture time from 1 hour to 50 minutes, scheduling break time between each lecture, and minimizing scheduling consecutive lectures. It is likely that, as we return to in-person and/or hybrid learning formats, screen time will decrease naturally. Second, the evaluations on the quality of the professional network formed during remote-learning SSP were not as high as other metrics (Figure 2D). This was likely due to the constraint of the remote-learning format and the lack of opportunity for in-person interaction. We therefore recommend building in structured, informal networking time for students to better establish and maintain networks with their peers and members of the division. Another challenge was the burden of lecture planning that fell on members of the division with little preparation time. To assuage some of this responsibility, we recommend taking advantage of outside resources to supplement the content that instructors create. This saves instructors time and can also reframe information, which can be helpful to students across different learning styles. To this end, we worked with the Northwestern Galter Health Sciences Library and The Graduate School, which sponsored seminars and workshops that they opened to our summer students. We also sourced videos that we used to supplement lecture information. This approach created a “flipped” classroom to accommodate those who learn better from Socratic discussions than from traditional lectures. This allowed students to move at their own pace and removed the burden of lesson planning from mentors.

We have several recommendations for other institutes that plan to implement similar programs. First, universities should provide stipends to those participating in their SSPs, even if they are held remotely. For many students, receiving a stipend is critical for their ability to participate in these programs. Therefore, although the pandemic has decreased operating budgets at many higher education institutes, SSP budgets should be prioritized. Second, we recommend taking an active approach to cultivate community between participants. This includes building in time during which students can bond among themselves without members of the division present. Finally, we suggest gathering anonymous feedback weekly. This can help catch problems early; for example, a short survey can reveal if students are overloaded, understimulated, or experiencing screen-time burnout.

In summary, we have created a remote-learning model for an SSP. This model leverages the knowledge base of many different division members, including research technicians, trainees, and faculty. Although the COVID-19 pandemic has drastically changed the landscape of higher education and laboratory-based research, there remain viable ways to maintain student engagement in SSPs. Although we do not yet know our model’s long-term success in retaining students in STEM, our metrics indicate that the program was successful in the short term.