Discovery: Virtual Implementation of Inquiry-Based Remote Learning for Secondary STEM Students During the COVID-19 Pandemic.

Challenge Statement

Discovery is an inquiry-based STEM educational program offered free of charge by the University of Toronto in partnership with the Toronto District School Board. The program exposes senior secondary school students to scientific thinking and engineering design through engagement in capstone-style projects within on-campus teaching labs, under the mentorship of undergraduate and graduate students. Toward these goals, engagement of students from participating schools is a mandatory component of university-preparatory grade 11 and 12 biology, chemistry, and/or physics classes. By program design, mandatory engagement is used to engage capable students with initial reticence to participate in extracurricular STEM activities due to social, academic, or financial factors; the marks assigned for Discovery deliverables comprise 10-15% of the respective final course grade. To date, we have observed benefits to engagement and interest in STEM for students participating in this learning model, built from previous evidence-based practices for optimal inquiry-based learning experiences.1

The onset of COVID-19 and subsequent cancellation of regularly-scheduled on-campus Discovery activities in Spring 2020 provided an opportunity to pivot learning objectives to the virtual environment. The instructional team sought to develop challenging yet feasible virtual projects in keeping with the experiential learning of Discovery. Emphasis was placed on critical thinking and quantitative analysis, with maintenance of the degree of autonomy and creativity inherent to a traditional engineering or science pursuit. As a result of partnership with local schools, Discovery content must meet Ontario STEM curriculum learning outcomes and promote development of scientific investigation and career exploration skills. Spring 2020 presented an additional challenge to program engagement given that the Ministry of Education mandated that student course grades would not decrease, but could increase, from pre-COVID levels post-school closure. Although students were aware that Discovery engagement tends toward an increase in final grades,1 the challenge remained to ensure subject matter was sufficiently relevant and interesting for students to retain motivation and engage maximally with the program.

Novel Initiative

A thorough overview of the ‘normal’ Discovery workflow and timelines for one semester of in-person delivery can be found in previous communications.1 Briefly, students visit campus for 3 full days over the course of 6–8 weeks. During their first visit, they complete a procedure based “skill lab” that follows the typical undergraduate laboratory approach of a set protocol and analysis instructions. During the next two visits, students carry out independent open-ended design or experimentation projects under the guidance of undergraduate and graduate student mentors, who provide additional insight regarding post-secondary STEM education and career options. Discovery uses the Canvas web portal (Instructure, Salt Lake City, UT) to circulate project materials including protocols, literature, sample data, and software; students can also access a Discussion forum to consult with mentors throughout program engagement. The experience culminates with a symposium allowing students to present research posters and attend a keynote lecture.

Emphasis of quantitative investigation, a main program goal for the in-person Discovery framework, has previously been met with varying success depending upon subject and project. Physical experimentation has often dominated on-campus instructional time, thereby relegating analysis, interpretation, and poster creation (a.k.a. the “lab report”) to off-site completion with challenging access to mentor support. We therefore saw the pivot to an online offering this past semester as an opportunity to showcase how objective, quantitative methods can be applied within all STEM disciplines. A “data challenge” was presented that focused on quantitative analysis and interpretation of a provided (biology; chemistry) or student-generated (physics) dataset. Data was supplemented by information on how the techniques and concepts applied to current research and healthcare efforts in the COVID-19 pandemic, providing real-world context in learning. To foster interest and skill development relevant to the data challenge, students were also provided with a skill lab analogue (physics) or virtual learning environment (VLE) replacement (biology; chemistry) that demonstrated how these data are collected, as well as the goals of subsequent analysis. All educational materials, found in the Appendices, are summarized in Table 1.

Table 1

Discovery curriculum for each STEM subject.

Subject	Grade	Labster simulationsa	Data challenge	Curriculum outcome focus (grade)
Biology	11	-Lab safety -Pipetting -Mendelian inheritance -Animal genetics	Students were provided with a biology pedigree chart, sample gel images of PCR products, and access to open-source image analysis software Using these resources, students were asked to identify the method of inheritance and risk to family members of two pathologies	Genetics (11) Molecular genetics (12)
Chemistry	11	-Lab safety -Pipetting -Solution preparation -Stoichiometric calculations -HPLC	Students were given the measurements from chromatography experiments performed with different protocols for separating a binary mixture of proteins, as well as the data corresponding to a calibration curve to estimate protein concentration. Students were expected to determine the equation of the standard curve and subsequently calculate the concentration of protein in each fraction of the chromatography experiments in order to determine protein separation efficiency and yield	Structure and properties of matter (11) Organic chemistry (12) Biochemistry (12)
Physics	12	-Conservation of energy -Law of universal gravitation -Electromagnetic spectrum -Wave model of light	Part 1: Students were provided with a set of accelerometer readings captured during different movements. Students analyzed data and matched data to descriptions of each movement Part 2: Students used a smartphone application to collect acceleration data during two or more states (e.g., deep breathing and coughing). Students analyzed data collected from each group member to identify distinguishing signal features in each state	Kinematics (11) Forces (11) Dynamics (12)

aLabster simulations were not available to complement the subject matter of the data challenge in the Physics course. We instead chose simulations that complemented the Ontario course curriculum

The term “virtual learning environment” (VLE) can define use of a simple learning portal with access to supplemental materials and discussion board capability, up to and including virtual “gamified” labs. For even relatively simple VLEs, increased mean daily activity has been positively correlated with performance, especially in STEM programs.2 In our COVID-19 Discovery offering, the VLE consisted of a highly-functional portal allowing student group formation, discussion, and data sharing, as well as video conferencing and assignment feedback. Program design also leveraged Labster simulations (Labster ApS, Copenhagen DK), a virtual lab platform with diverse STEM modules, as a means of demonstrating data derivation in multiple forms and for facilitating connection of classroom concepts to the open-ended nature of the initial raw datasets. Certain Labster simulations have been shown to significantly enhance outcomes of students’ understanding compared to a passive educational model.4 When compared to experiences in laboratory-containing courses, students in courses that included Labster simulations reported a significantly higher level of confidence in performing technical tasks.3 In this study, there was no significant difference reported in knowledge development in course material compared to courses with standard laboratory modules. In general, virtual simulations have been found by meta-analysis to be effective at enhancing learning outcomes relative to various control groups.5 This suggests that Labster simulations may represent an appropriate situational surrogate for traditional guided laboratory experience when there is no physical alternative.

We included Labster for the first time in our curriculum to illustrate a typical lab workflow, and to provide visualizations for traditionally difficult concepts in each subject. In the context of Discovery, Labster also helped students interpret data presented in the data challenge, and provided context for how a quantitative dataset was generated for each subject. Post-engagement with Labster simulations, each project progressed to a data challenge with clearly-stated expectations for completion, using fictitious but complete datasets (or user-generated, in the case of Physics), within our Canvas-based learning management system. Labster simulations that best fit intended curricular concepts were selected for student use; for example, the HPLC simulation demonstrates similar content and workflow to the Chemistry anion-exchange protein chromatography topic, but requires specific considerations for data analysis and interpretation (Appendix I). Although not ideal when compared to the value of iterative in-person skill development, Labster simulations represented an appropriate surrogate because, in contrast to video representations, simulation allows students the capacity to perform actions and learn by failure. Additionally, students were supported by weekly mentor meetings (one per week of engagement) that ranged from 15 to 75 min, depending on student need.

Reflection

In delivering a virtual Discovery offering, we retained our main program goal to stimulate high school student critical and objective thinking using active learning. Reflection of 7 Discovery mentors and three participating secondary school teachers who directly oversaw their students’ participation was garnered through exit surveys and structured interviews, respectively, and demonstrates this pilot online initiative to be a success (Table 2). Over the course of the program, we observed online engagement from a majority of participating students. Holistically, this suggests that our online delivery strategy was successful in engaging students in a meaningful way despite previously identified challenges. The Physics program was deemed the most successful, both in terms of student engagement and performance, as attributed by mentors and teachers alike due to its retention of the inquiry-based, open-ended curriculum strategy used in previous semesters. The use of a discrete, directed activity to introduce skills, as well as the ability for students to perform proper hypothesis testing through their own data acquisition, were highlighted as key success points. These factors were attributed by both mentors and teachers to be key for both student engagement and understanding, given the open-ended context of the program. For all subjects, the end-of-term symposium featured webcast slide show presentations that showcased student understanding as well or better than poster presentations, by mentor and teacher perception.

Table 2

Teacher reflections regarding the virtual Discovery inquiry-based secondary STEM program.

Topic	Biology	Chemistry	Physics
Overall impression of virtual offering	“Normally biology is make up your own procedure. And that’s what kills the kids; the anxiety over making a procedure and testing to know the lab for two solid days. [But virtually], the quality was still really high. The students started using [Canvas] for the first time; they’ve been given [Canvas] before, and they’ve never engaged with it. This time students really engage with Quercus”	“The pandemic teaching experience was like a virtual classroom, plus we had virtual Discovery on top of it. Maybe [the students] were overwhelmed. I don’t blame them. I was myself overwhelmed”	“Without the open-endedness [of in-person offerings], students were more directed. So they were told what to do what they had to assess, what they had to analyse, and the data sets they had to get. When we were actually in the labs, we had a diversity of which way people wanted to go, and they realised that actually wanted to use different equipment than other groups; they could see the other groups doing other work”
Critical thinking and open-endedness	“It was more application of knowledge than being completely open ended like last semester. If it was completely open ended, I’m not sure we would have had as much success as we did this semester”	“[Using] equipment, working together with mentors, and getting first-hand experience is very helpful to students but, virtually they did not have that experience. [In terms of] the background information that was given to them; they were […] reading and getting to know what they are going to be dealing with. And in terms of questions, it was good; it helped them critically think much better”	“In terms of critical thinking skills, I think there was less because it had to be more directed. And the skills involved were more they were led into what [the students] had to report and analyse. However, I still think they were engaged. I think the quality was still there. I think they still gained the experience”
Student engagement	“For the students that engaged in it, they made sure [that their performance] was really high. The only problem this year is students didn’t have to engage So, I’ve have had three students that did not engage at all while the rest did engage. It was noticeable for the students that were with lower marks; they definitely put a lot more work into it. And they ended up with higher marks. One student in particular, in class, [does not show] a lot of engagement; his work always comes in late, and is fairly low quality. This time he did really well”	“Out of 18 students, [for] 5 of them the engagement was very minimal. And I am not sure if every student participated equally; if one of them maybe took over all the work. But I think they mostly worked hard”	“Even before [COVID-19 shutdowns] started, they knew [Discovery] was coming. When I sent out the message through Google Classroom that we’re still doing Discovery, then the panicked messages came in. At the same time they’re panicked, but they’re excited. But everyone who would have participated in-person participated virtually”
Quality compared to in-person offering	“This semester, it was not [as open-ended]; it was very prescribed. But the learning was still there”	“First of all, in terms of virtual, of course they had a chance to do some of the things virtually and they did not have a chance to have the first-hand experience in the lab. But still, it’s a good chance for them that means sometimes in drastic situations you can do certain things virtually to and get the experience” “One of the advantages of Discovery I am looking at is the laboratory experience … As I said for me, Discovery means firsthand lab experience and using the equipment”	“I think it took time. I think the process of understanding the scope of what was expected of them took more processing time then had they been [on-campus] and it be explained to them personally”
Labster simulations	“I think the students probably enjoyed it because it’s kind of like a computer game it’s something different. And those were the [deliverables] that came in really fast. I think the students found it engaging and easy because all they had to do was go through them. How much did they actually learn on it? I can’t tell you but I know all the students did all the Labster simulations”	“And of course, those modules helped them a lot. But still, as I said, virtual had lots of challenges”	“And I highly encourage [using the simulations] in the future. Even if we go back to the physical learning setting, I would encourage that we still have some Labster modules or some online experience for the students”
Virtual symposium	“My VP and principal were sitting in on [the talks], and they were just blown away by how well the kids are doing. I did like the concept where it was a biology group, a chemistry group, and a physics group, and they’re presenting to their peers [However] I think there was a lot more focus being paid to different things, compared to a big symposium. You can’t replace the experience of a big symposium”	“The virtual presentation was not as engaging [as in-person]. I did not like as much as I should have liked it. But the students tried a lot. I mean, they did so much work”	“I think they all took it seriously. I think they were all intimidated to try their best. And I think they all put forth effort. I think, you know, there were there are issues [concerning confidence] On the other hand, I felt that there were others who normally are very shy in person, you know, did very well. And there’s one particular student that I was in awe of, […] because in class, he’s super shy. And in this presentation, he was answering all the questions; his voice seemed confident […] Students had the experience of presenting, which honestly is tremendous for them, because that gives them a sense of accomplishment. It teaches them how to handle themselves in a question period, and it just gives them more confidence in themselves”
Most critical point for improvement	“Normally I am a lot more engaged with my students with the Discovery work because I am at the labs with them. I can sit and talk to the kids, I can see what they are doing, all this kind of stuff which obviously wasn’t possible this time. If somehow, it becomes more open ended [like when] we are in class, then that gets the teacher back to engaging with the kids. You can start to see the learning process. You can start helping them out; you can get more on top of them […] Having the teachers stand in front of the students saying ‘Where is your work, I need it now’ is a lot more effective than emailing students saying ‘Please give me your work.’ It just works out that way”	“The pandemic limited the time we had, and virtual learning is a new experience for all of us. But ideally, the work would be spread over longer, and the mentors would be able to check in over longer periods”	“I found that there was less flexibility for them to create and focus the project”

Teachers further reflected upon the benefits of continuing the program despite inability to work on-campus; these included a sense of continuity, the application of concepts in a manner that requires deep understanding, and the importance of quantitative analysis. In general, students demonstrated keen interest considering their low level of familiarity with the content, the methods of instruction, and the analytical techniques used. Nevertheless, although the program was by all accounts impactful on student engagement, we have identified several areas of challenge that can be improved for future virtual/hybrid offerings.

Teachers reported consistent levels of engagement from individual students who would normally be highly engaged in the on-site program. However, there was a reduced degree of engagement in the virtual offering from students with lower initial interest or drive, despite the mandatory nature of the program. We attribute this change in engagement to three main factors. First, government mandated educational policy indicated that final course grades could not decrease from pre-shutdown levels, which gave incentive for some students to participate only at a basal or noncommittal level. Secondly, certain students may have participated at a lower level due to health, family, employment, or technology access concerns. Finally, the lack of open-ended investigation in the biology and chemistry projects may have failed to optimally motivate students; in contrast, the physics project was deemed a success and mentors further reported that students included their families in data collection. Nevertheless, the focus in all subjects to find a correct answer may have discouraged students from exploring or experimenting in a manner similar to that previously observed in the physical laboratory. Considering content, Labster simulations were found to be important for bridging comprehension of relevant conceptual topics, understanding how relevant measurements would be conducted and how data would be gathered, and interpreting discrete results in the context of the simulated experiment. However, our interviewees suggested that simulations were not especially useful in preparing students to conduct more-involved quantitative analysis or interpret datasets in a more open-ended context, as those skills were not explicitly used in the simulations. Furthermore, although a significant amount of custom instructional video was generated, many students did not access this material or demonstrate comprehension before meeting with their mentors. This suggests the need to better design the relationship between students and mentors at program onset to ensure students independently pursue available resources with limited guidance.

Groupwork was also identified to be more challenging in the remote setting than in the physical laboratory. Mentors and teachers highlighted difficulty in attributing individual student project contributions, especially given the low amount of student/mentor contact time when compared to a typical offering. The heterogenous nature of student contributions was attributed both to technological connectivity issues, as well as differing motivations and feelings of responsibilities between students.

Consequently, the student-mentor connection was found to be especially challenging to recapitulate in an online setting. Scheduled weekly meetings were ubiquitously less productive at onset of term, and appeared to be of value to students once they had engaged with the materials and become more familiar with their mentors. In most meetings, time was dominated by reviewing logistics (e.g., navigating the web portal, accessing educational materials, reiterating assignment instructions) and encouraging familiarity with the material. Although pre- and post-meeting quizzes were provided to focus student questions and summarize next steps, respectively, these often went uncompleted and therefore did not provide mentors with clues as to student need or progress. Similarly, mentors reflected that even during meetings, it was difficult to establish the aspects of the project (in terms of workflow or conceptualization) that were challenging to students, and the specifics of mentor assistance or intervention that would be helpful. Students also used the web portal discussion boards less than in previous semesters, further limiting their contact with mentors. Finally, student engagement metrics (e.g., page views) are available for both Labster and the web portal. These have been previously tracked for analysis in pedagogical study,2, 3 and in the absence of obvious student cues, may have been useful in allowing Discovery mentors to better assess student engagement and progress.

In summary, we believe that although the virtual offering of Discovery succeeded in engaging students and promoting STEM interest during a semester that would otherwise fail to meet curriculum guidelines, there are a substantial number of concrete improvements that can be made to the program in future semesters. These changes reflect a return to our previous open-ended model that is heavily reliant on student inquiry, iteration, and mentor interaction. For this reason, the virtual Physics module will serve as a model for all of our virtual and/or hybrid offerings moving forward. All virtual Discovery offerings will be structured using the following identified central themes:

Implementation of a more inquiry-focused pedagogical strategy: The Fall 2020 semester will return to an open-ended inquiry-based structure, supporting student creativity and interest in both posing an addressable question, and developing a design or experiment with which to address it. The data challenge will involve simulated raw data (instrument readings, micrographs of cell culture, etc.). Students will request experimental data based on their plan and the imposed conditions in iterations using a standardized form delivered to the Discovery team, or perform said virtual experiments themselves by inputting experimental parameters into a closed model using a graphic user interface. By either method, the resulting data will include built-in variability, necessitating proper planning and interpretive care. Further, despite the technical ease to acquire data, students must justify their experiments. Therefore, analysis will be needed at each step to assess data, which then poses a natural limitation on being able to exhaustively run every potential experiment.

Optimization of content clarity: The content of the skill lab, introductory lecture, and reading materials will be expanded, ordered, and made more relevant.

Development of extended and discretized scheduling: Students struggled with both a short and excessively open-ended schedule in this offering. The project length will be extended to 6-8 weeks to reduce student stress and allow flexible access to materials; however, incremental milestone deadlines will be used to ensure regular student engagement. Returning to the previously optimized Discovery framework, a request-for-proposals will be presented and student proposals will be an evaluated deliverable. A virtual skill lab will be presented, using a guided workflow and clear objectives, to introduce students to the analysis workflow needed to complete the data challenge.

Improved integration with VLEs: Labster or an analogous simulation platform will be incorporated, but with greater effort to connect the themes and internal workflows with project content to clarify the relevance of the VLE to students. Consistent with the need for enhanced student scheduling, simulation completion will be the first milestone deliverable in the program.

Increased frequency and number of mentor connections: More mentor meetings, with expectations of progressively longer meetings over the program course, will be implemented. Student-mentor rapport will be specifically targeted using an icebreaker activity during the initial lecture. To better ensure action items result from interactions, greater structure will be given as to meeting format by program leadership. Specifically, mentor meetings will be scheduled to allow for interpretation and discussion of a fresh data set, and to establish a plan for future work.

We believe that circumstances surrounding the changed COVID-19 learning environment may offer a unique opportunity to engage driven students in open-ended learning. These specific program improvements to be made in upcoming virtual Discovery offerings can be generalized into four guiding principles that can be applied to other open-ended learning programs:

Secondary students do not require full autonomy in scheduling of a large project. Inclusion of detailed scheduling milestones and assessment/deliverable details at program outset provides sufficient guidance for effective student progress. Further, this introduces accountability to the project and liberates students to allocate greater mental energy to open-ended thinking in line with program content, rather than repeatedly reassessing an appropriate outline.

In group-based online projects, students should be encouraged to establish team roles and provide support for intra-group scheduling to improve overall group function and accountability. We found that partnerships worked better than groups of 3–4 students when considering the additional challenges of scheduling and workflow in the remote setting.

Regular, short check-ins with mentors should be scheduled to allow for the early detection of challenges (i.e., motivational, academic, or personal). Regular contact empowers students to ask more detailed questions about program content. Students who struggle with mastery-based content may perform very well with more autonomy over their project, provided they are given the encouragement and support as needed.

A measurable endpoint to project-based learning must be included. It can be logistically challenging to implement student presentations in a remote-learning environment, but we feel that this type of end-point gives validation to data challenges that otherwise might be difficult to contextualize in the absence of iterative engagement within a lab setting.

With this revised structure in place, we are optimistic that our upcoming offering in Fall 2020 will better fulfil our aims of relevant content application towards an inquiry-based project, with meaningful mentor-student interactions to better perceive STEM opportunities. This framework can be used across the range of STEM disciplines; guided ownership of an open-ended project is likely to be helpful for any high school class. This could also conceivably be implemented as a permanent fixture in university-empowered educational partnerships, such as in collaboration with remote or underfunded communities.

Below is the link to the electronic supplementary material.

Electronic supplementary material 1 (PDF 1074 kb)

Electronic supplementary material 2 (XLSX 12 kb)

Electronic supplementary material 3 (PDF 5580 kb)

Electronic supplementary material 4 (XLSX 3087 kb)

Electronic supplementary material 5 (PDF 843 kb)