U.S. and Finnish high school science engagement during the COVID-19 pandemic.

When the COVID-19 pandemic struck, research teams in the United States and Finland were collaborating on a study to improve adolescent academic engagement in chemistry and physics and the impact remote teaching on academic, social, and emotional learning. The ongoing "Crafting Engaging Science Environments" (CESE) intervention afforded a rare data collection opportunity. In the United States, students were surveyed at the beginning of the school year and again in May, providing information for the same 751 students from before and during the pandemic. In Finland, 203 students were surveyed during remote learning. Findings from both countries during this period of remote learning revealed that students' academic engagement was positively correlated with participation in hands-on, project-based lessons. In Finland, results showed that situational engagement occurred in only 4.7% of sampled cases. In the United States, students show that academic engagement, primarily the aspect of challenge, was enhanced during remote learning. Engagement was in turn correlated with positive socioemotional constructs related to science learning. The study's findings emphasise the importance of finding ways to ensure equitable opportunities for students to participate in project-based activities when learning remotely.

The 2019–2020 school year brought significant changes to educational systems around the globe when elementary and secondary schools closed suddenly, finding themselves faced with new social distancing guidelines as the world plunged into a crippling pandemic (Meluzzi, 2020). According to the United Nations Educational, Scientific, and Cultural Organization (2020), these closures impacted over 63% of students enrolled in pre‐primary through tertiary learning institutions worldwide. Schools closed with little or no notice, leaving parents and educators barely time to prepare for this new reality. With the shift to full days of remote instruction, teachers and students found themselves adapting to entirely new learning environments.

Unfortunately, more technology does not imply improved educational outcomes (Escueta et al., 2017). With many students on Zoom or other platforms, equitable participation became a serious problem during the pandemic. Domina et al. (2021) showed students' academic engagement was improved with greater access to technological resources and quality instruction that included socioemotional learning. Globally, however, socioeconomically disadvantaged students are less likely to have the tools they need to participate in remote instruction (Meluzzi, 2020). Students in lower‐income schools were shown to be less engaged with their schoolwork than their same aged peers with greater access to resources (Hopkins et al., 2021). Learning losses are, in turn, shown to vary with academic engagement and access to school supplies or technology necessary for participation (Dorn et al., 2020).

The inability of some families and schools to provide the financial and material support for experiences that would normally be provided in a science classroom means existing gaps hindering equitable participation are exacerbated by the pandemic. In the United States, for example, nearly one third of students were unable to participate in remote learning during the first wave of the pandemic (Meluzzi, 2020), placing additional stressors on students and their families. Not surprisingly, the pandemic has coincided with an increase in student anxiety, which inhibits students' ability to engage with their online classrooms (Yang et al., 2020). High school students are experiencing stressors of the pandemic, many of which limit the coping mechanisms teenagers usually employ to deal with the normal stressors associated with being in high school. Survey data has shown students reported feeling disinterested, bored, and socially isolated when spending long hours in virtual classes; parents also have expressed similar concerns about their children's academic learning and well‐being (Kaufman et al., 2020).

Recent literature shows the pandemic has caused difficulty in attaining academic engagement in remote classrooms with many teachers reporting the need for additional resources to do so (Trinidad, 2021). Literature regarding learning during the COVID‐19 pandemic suggests that students' academic engagement may differ when content is delivered remotely as compared to learning in a classroom environment. In a joint project of two countries, the United States and Finland, we aim to better understand academic engagement and its correlation to various activities assigned in science classes during remote learning due to the pandemic.

A TWO‐COUNTRY INTERVENTION FACING A PANDEMIC

The sudden transition to remote teaching occurred during an ongoing collaborative intervention, “Crafting Engaging Science Environments” (CESE). Funded by the National Science Foundation and Academy of Finland, CESE brought together a team of learning scientists, science education researchers, psychologists, sociologists, and teachers. They designed an intervention that supported students' academic engagement and impacted not only their academic learning, but also their social and emotional learning (Schneider et al., 2020). Based on the principles of project‐based learning (PBL) that support student experiential activities in “figuring out” phenomena (Krajcik & Shin, 2014), the unifying theme that motivated the CESE intervention was to improve engagement in physics and chemistry courses for high school students in grades 10 through 12. Throughout multiple years of this collaboration, a series of questions related to the conceptualization of engagement and its impact on social and emotional learning remained a continual focus.

During the 2019–2020 school year, CESE was in its first‐year efficacy trial in Finland. In the United States, CESE was undergoing a maturation study. Having shown promising results for the learning outcomes of treatment students in the previous year (Schneider et al., under review), the goal was to determine whether teachers in their second year of teaching the project‐based learning intervention would show greater impacts on learning outcomes than teachers implementing the lessons for the first time. Although the pandemic ended the ability to study students' engagement during hands‐on lessons, a unique opportunity arose to study academic engagement in this remote learning environment.

U.S. and Finnish government responses

CESE's shifted focus to studying academic engagement during remote science classes occurred within two contrasting national contexts. The two countries saw differences in the transition based on their respective approaches. The relative populations of the United States (over 330 million) and Finland (over 5.5 million) affected each country's response to the pandemic. Finland was able to centralise its decision‐making, given its smaller population. Centralization in the United States was more difficult, not just because of its larger population but each of the 50 states has the right to control its own schools. When U.S. schools closed, teachers in the CESE study reported guidelines for learning and instruction that differed among states, districts, and even schools.

In the United States, the movement to remote instruction began in mid‐March, which coincided with spring break in many districts. Assuming social distancing measures would be brief, some schools simply extended the spring break. Awaiting guidance from the state or federal governments meant many classrooms did not make this transition until April. Consistent with the findings of a 2020 study from Reich et al., policies differed at multiple levels of decision making; some CESE teachers reported that their schools required all teachers to use the same curriculum and had strict protocols for contacting parents, while other teachers reported that their schools entrusted teachers with all instructional and logistical decisions. Schools, teachers, and students had limited familiarity with remote learning. Orchestrating an equitable learning environment where all students, especially those in low‐income families and communities, were equipped with computers and internet access was a monumental undertaking. When districts were unable to provide equipment for every student, they had to develop alternative methods that assured equitable learning experiences.

Planning and organising some form of high‐quality remote instruction for the 50 million primary and secondary students in U.S. public schools became exceedingly challenging. Despite the confusion surrounding these unprecedented changes, some national studies suggest that the majority of teachers and administrators reported effective communication from their districts regarding policy changes and that instruction was supported through relevant professional development opportunities (Kraft & Simon, 2020). Consistent with these findings, teachers in the CESE study noted that their districts provided professional development opportunities focused on adapting their pedagogy with a variety of online tools.

In contrast to the United States, Finland's government was able to quickly decide that all students in the country would transition to remote learning. All schools were closed from March 18th until May 13th. For students in upper secondary schools, vocational training institutes, tertiary, and other educational institutions, the government recommended continuing distance teaching until the end of the semester. The government worked with the schools to ensure that all students and teachers had access to computers, wi‐fi, and instructional guidance for students and staff (The Finnish National Agency for Education, 2020). Several educational platforms were already in place and allowed educators to provide feedback, assign homework, and communicate with parents and students through the pandemic.

According to the Finnish Teachers' Union Survey, about half of the teachers reported having sufficient pedagogical and digital competence for teaching during the remote learning period; a similar proportion of students claimed the change to distance learning went well. Only a small number of students had difficulties due to insufficient equipment or lack of skills for distance learning. Overall, Finnish students and teachers both reported having considerable experience with digital skills such as how to use computers, how to find information, and using reputable sources to “fact check.” These survey reports are consistent with several other international studies which have shown that Finnish students are among the most well‐prepared to work online compared to students in other industrialised countries (see, Fraillon et al., 2019, International Education Association (IEA) International Computer and Information Literacy Study). Not surprisingly, Finnish teachers and students reported feeling quite confident about their abilities to succeed in distance learning and had the skill sets to do so.

STUDYING ENGAGEMENT

Even before the pandemic, students' academic engagement in science was a deep concern globally, and several major reports by OECD (2019) connected engagement with interest in science and its attractiveness as a career option. One of the major challenges of CESE was to theoretically describe and measure academic engagement in science classes. Part of the problem was that while there was consistent agreement that academic engagement varied over time, what engagement meant in various contexts was viewed differently. One of the first considerations was how to specify academic engagement and what constructs should be used to define it (see, Hidi & Renninger, 2006; Schneider et al., 2016).

CESE views academic engagement in science as being comprised of interest, skill, and challenge (Schneider et al., 2016): and not all activities are likely to have the same effect on students' social and emotional or academic learning (Inkinen et al., 2020). Recognising the difficulty of trying to define academic engagement without specifying when it occurs misses the ability to identify when students are feeling interested, skilled, and challenged in what they are doing. The approach identifies these three constructs as critical for enhancing students' academic engagement which are grounded in psychological literature. Interest is the psychological predisposition for a specific activity, topic, or object; skill is the mastery of a set of specific tasks; and challenge is the willingness to take on a difficult, somewhat unpredictable course of action. When students report high interest, skill, and challenge, they are considered to be engaged.

Situational engagement/optimal learning moments

The primary focus of the CESE study is situational engagement. When measured in the moment, instances of situational engagement are considered optimal learning moments (OLMs), which are situationally specific times when a student is so deeply engrossed in a task that it feels as if time flies by Schneider et al. (2016). During those times, students tend to be concentrating and feeling in control (see, Salmela‐Aro et al., 2016). This idea is similar to how Csikszentmihalyi (1990) describes flow as being completely immersed in an activity. For this study, we consider OLMs to be situations that elevate students' academic engagement and are positively related to social and emotional learning. Our research shows that OLMs occur about 15–20% of the time in science lessons (Inkinen et al., 2020; Schneider et al., 2016). Our interest is to examine how often they occur when students are learning remotely.

Academic engagement and social emotional learning experience

Researchers recently started to examine the relationships between academic engagement and other factors including social–emotional skills and learning experiences as they mutually reinforce one another (Salmela‐Aro & Upadyadya, 2020). The current study applied the OECD framework on social and emotional constructs which include maintaining positive emotions, managing social relationships, and keeping goal pursuits. Some key elements are optimism (ambition or future importance), persistence, curiosity, social interaction, and self‐efficacy (Kankaras & Suarez‐Alvarez, 2019). In this study, we aimed to understand social–emotional experiences and their relationship to academic engagement during remote learning.

One important social–emotional learning experience is persistence, also referred to as grit (see Duckworth, 2016; Tang et al., 2019), which refers to how long students stay with a task or learning assignment without giving up. In relation to our idea of academic engagement, it is important to determine whether students “give up” in more challenging situations and whether “grit” acts as a buffer, inspiring students to persist in the task at hand (Salmela‐Aro & Upadyadya, 2020; Tang et al., 2021). “Grit” has particular importance to Finnish society as it has been associated with the term “sisu,” which can be translated as “determination to overcome adversity,” and is a hallmark of the Finnish perception of their national character.

Another social–emotional concept related to academic engagement is curiosity, defined as the desire for knowledge or information found to be associated with question‐asking, exploration behaviours, and achievement (Hidi & Renninger, 2006). Recently, curiosity, the epistemic emotion that triggers interest, has also been highlighted in the engagement process (Hidi & Renninger, 2019). Thus, the role of curiosity in OLM is important to investigate.

When students are engaged in learning we expect them to feel that their science work is related to their future education goals (Schneider et al., 2016). U.S. students who participated in the CESE intervention during the previous year's efficacy trial showed increased educational ambitions (Schneider et al., under review). During a pandemic, however, uncertain futures might weaken educational goals. Alternatively, science is now trending heavily in the news which could foster students' curiosity and engagement with it, in turn strengthening their educational ambitions.

Current study

When the pandemic struck, CESE teachers in both countries were unable to teach the intervention. Although CESE was rendered unable to study students' academic engagement during these project‐based lessons in a science classroom, a unique opportunity arose to study engagement in relation to the activities assigned in this remote learning environment. As mentioned above, comprised of three components (challenge, interest, and skill), it was possible that academic engagement could increase or decrease based on these subsequent parts. If students felt more challenged in the remote environment or more interested in science due to ties to current events, it could increase their academic engagement. In Finland, CESE researchers sought to understand situational engagement during remote lessons. Without the clear project‐based learning features, would students be less engaged during remote lessons?

In the United States, where the transition to remote teaching varied greatly among schools and districts, materials distribution to study situational academic engagement was not possible. Instead, students were surveyed during the pandemic, and results from those who responded were compared to their responses collected earlier in the year. In Finland, the same students were not tracked over the course of the school year due to the semester‐by‐semester class structure of Finnish high schools. Here, the more organised shift to remote teaching made it possible to provide students with a diary to collect data about their engagement in the moment. Despite their differences, the study teams were able to collect similar data related to academic engagement in science. Guiding the investigation at this unprecedented time are three major questions: (a) how engaged were students in their remote science classes? (b) how engaged were students in their specific learning activities during remote learning? (c) how was academic engagement related to social and emotional learning experiences during remote learning? This is not a comparative study; rather, its intent is to underscore similarities and differences in students' academic engagement when in remote science classroom environments.

METHODS

U.S. and Finnish samples

The ongoing work in both countries allowed CESE to survey and interview the students and teachers when they were participating in their remote science classes. To measure academic engagement, we have deliberately selected academic and social–emotional constructs that both countries have investigated with their own respective student populations with some variations and differences in instruments which are explained below. Both countries used self‐report surveys to collect information from high school students enrolled in chemistry and physics courses. In the United States, data were obtained from students in fall 2019 and then again during the pandemic, allowing for comparisons of attitudes and perceptions from in‐person to remote experiences. In Finland, however, data were only obtained in the spring.

U.S. sample

During the 2019–2020 school year, the United States was in a maturation study where both treatment and control teachers from the previous year taught the intervention, for their second and first years, respectively. In the beginning of the school year, 4954 high school students from 86 physics and chemistry classrooms completed a background survey as part of the CESE intervention. In the United States, the expected age range of students in grades 10 through 12 is between 14 and 17 years old. In the CESE study, 95% of students fell within the anticipated age range and 98% were in grades 10 through 12. We initially anticipated that our response rate would be similar to the analytic sample from the efficacy trial in the previous year. Given the pandemic, however, it became apparent that we would not get anywhere near that percentage of respondents. Taking into account attrition rates from the previous year, teachers' reports of low attendance after the shift to remote learning, the loss of one highly populous district, and district policies that gave little incentive for students to attend remotely, we expected to receive less than 20% (n = 879) of our original 2019–2020 sample. When surveyed again in Spring 2020, 922 students replied to the exit survey. Of the students who responded to the exit survey, 81% (n = 751) had completed both a background and exit survey in the participating regions. These students were retained in the analytic sample, of which 55.53% are female; 22.64% white (non‐Hispanic); 40.21% Hispanic; 5.73% Black; 6.39% Asian, 17.04% multiple race/ethnicities; less than 1% other, and 7.46% did not provide information about their race or ethnicity.

Finnish sample

When the pandemic struck, the Finnish team reached out to six of the nine participating teachers involved in the efficacy study regarding the possibilities of collecting data while students were learning remotely. All six teachers agreed to the study, which included 203 students (97 males, 103 females, 3 who preferred not to answer; all within the range of 16–18 years old). It is important to note that the number of students included in this study is relatively small, but all were in grades 10 through 12 and participating in the CESE intervention.

Measures

U.S. measures

In the United States, the same students were followed through a full school year and surveyed in fall 2019 and again in spring 2020, after moving to remote learning. Students were asked to respond to questions about their demographic data such as race, gender, grade point average, and attitudes toward science. In both fall and spring, students were asked about their interest, skill, and challenge in their physics or chemistry class. Students reported how much they agreed with the following statements: I am interested in science; I feel skilled in science; and I find science challenging on a four‐point Likert scale. To measure students' academic engagement, we calculated the mid‐point for each of the three categories (i.e. interest, skill, and challenge). Academic engagement is the binary outcome indicating that a student reported scores above the midpoint (e.g. 3 or 4) for each of the three categories. We then compared their responses from fall to spring after the shift to remote learning.

Students also reported on how frequently they participated in a number of activities when in a remote classroom and how interested they were in each activity. These activities included: discussion boards; one‐on‐one video chats with the teacher; watching videos of experiments; online simulations; live lessons; recorded videos of lessons; using textbooks; writing papers; building models at home; making presentations using slides or power‐point to share with the class; text‐based instruction; working in groups through video chat; and experiments to try at home. Students' reported interest was ranked and compared to its reported activity frequency in the classroom. Frequency was measured on a 5‐point scale: we do not do this activity (0); less than once every 2 weeks (1); once every 2 weeks (2); once per week (3); or every day (4). Interest here was reported on a four‐point scale ranging from “this does not interest me (1)” to “this interests me a lot (4).”

On the fall and spring surveys, students responded to questions specifically related to project‐based learning tasks associated with the CESE intervention. These activities were included to determine whether students were still able to engage in the same project‐based tasks fundamental to the intervention. These questions were altered to better suit the situation of remote learning when administered on the second survey and additional questions related to modelling were added. In fall, these were measured on a scale similar to those that were specific to online learning and where students reported the frequency with which they performed certain activities: never or almost never (1); once every month (2); once every 2 weeks (3); once per week (4); or more than once every week (5). These activities included several different types of modelling activities, opportunities to take pride in their achievements, ask questions in class, discuss phenomena, work together to solve problems, generally present their findings, and the frequency with which students performed “science and engineering practices like taking measurements to collect data about the world around us and using evidence to make a claim.”

To understand the relationship between social and emotional skills and students' academic engagement, college ambition was considered a behavioural measure of persistence (or grit). On both the fall and spring surveys, students were asked to report their educational goals regarding how far they expect to go in school, including: I do not know how far I will go; less than high school; graduate from high school but not go any further; go to a vocational, trade, or business school after high school; graduate from a two‐year college (Associate's Degree); graduate from a four‐year college (Bachelor's Degree); Master's Degree or Equivalent; and Ph.D., M.D., or other advanced professional degree. A binary variable was created to indicate whether or not a student reported plans to attend at least 4 years of college.

Finnish measures

In Finland, the CESE study focused on situational engagement and researchers did not follow the same students through an entire school year. During the pandemic, students were asked to complete two surveys. The first survey focused on their general feelings and experiences in remote learning during the pandemic. The second survey asked students to report their real‐time feelings and experiences using a diary format, the experience sampling method (ESM), for which they answered short surveys in the moment, during their remote lessons.

Situational engagement consists of high levels of interest, skill, and challenge. Students reported in the ESM survey their momentary interest (Are you interested in what you are doing?), skill (Do you feel skilled at what you are doing?), and challenge (Do you feel challenged by what you are doing?) on a four‐point scale of: not at all (1); a little (2); much (3); and very much (4). Academic engagement was measured similarly in both countries. Students were considered situationally engaged (i.e. OLM) if their responses were 3 or 4 to all three questions. A binary variable of 1 or 0 was generated to indicate whether this was an OLM or not.

The ESM survey asked students to report their practices when they received the survey. They could choose from: following teacher's instruction, doing tasks independently, studying from books, studying from the website, writing, discussing online, making videos, asking questions, developing a model, using a model, planning an investigation, conducting an investigation, analysing data, solving math problems, constructing an explanation, using evidence to make an argument, evaluating information, and other. Students chose all practices that applied to them. These options were recoded as dichotomous variables for the analysis (practice was reported (1) or not reported (0)). The ESM also surveyed students' social and emotional experiences in real‐time on a four‐point scale: not at all; a little; much; or very much. We focused on students' remote learning experience regarding their belief that the material had importance for their future; feelings of loneliness; boredom; confidence; curiosity; and grit as they have been highlighted in the OECD social and emotional skills frameworks (Kankaras & Suarez‐Alvarez, 2019; Salmela‐Aro & Upadyadya, 2020). In total, these ESM surveys produced an average of 3.49 responses per student, for a total of 701 situational responses.

The general survey asked students to report how often they engaged in the following science practices on a four‐point scale: never or hardly ever (1); some hours (2); most hours (3); and in all classes (4). The practices were: have opportunities to explain my own thoughts; plan how to study; do practical tests; draw conclusions from experiments or research; apply concepts related to everyday problems; participate in debate or discussion; follow the teacher's demonstrations; do experiments as instructed; follow the teacher's teaching or example in remote learning; view a video or animation; do assignments independently; study a book, study a website or e‐learning platform; make notes or summaries; share documents with other students; present my output in a video conference (Zoom, Meet, Skype, etc.); write a joint document with another student; chat online; create videos or animations; do experimental research with tools found at home; ask for advice from another student; help another student; and get feedback from the teacher that promotes learning.

RESULTS

U.S. results

How engaged were U.S. students when attending courses remotely?

Despite the many changes to instruction, U.S. students were more likely to report academic engagement after participating in the CESE intervention. Table 1 shows the change in the odds that a student reported above median scores for interest, skill, challenge, and the engagement variable, meaning they reported high scores for all three. As shown in Table 1, when surveyed in Spring 2020, students showed a strong increase in their science interest and the level of challenge they felt in their remote physics or chemistry class as compared to earlier in the school year. Students were 4.24 times more likely to report high levels of interest and 7.36 times more likely to report high levels of challenge. Students were only 1.53 times more likely to report high levels of skill during the pandemic. This change was significantly less (p < 0.001) than the differences in either of the other questions. The increase in all three categories resulted in students being 9.24 times more likely to be engaged.

TABLE 1

Changes in U.S. students' academic engagement during the 2019–2020 school year

	β	SE β	OR (eβ)
High interest	1.44 ***	0.13	4.24
High skill	0.42 **	0.12	1.53
High challenge	2.00 ***	0.14	7.36
Engagement	2.22 ***	0.19	9.24

Note. High Interest, High Skill, and High Challenge are binary variables indicating a student reported a 3 or a 4 on the scale. This table shows the change in the log odds of a student reporting high measures of these variables from fall to spring in the 2019–2020 school year.

*p < 0.05.

p < 0.01.

p < 0.001.

What kinds of activities were U.S. students participating in during remote teaching?

Figure 1 shows the frequency at which U.S. students reported specific class activities and how interesting they found these experiences during the pandemic. The most frequent activities used remotely were videos of experiments, online simulations, text‐based instruction, and discussion boards. Students reported watching recorded videos of lessons more frequently than attending live lessons. Using textbooks, building models at home, and making presentations were the least frequently used. While performing experiments at home was one of the top interests reported by students, the frequency at which that occurred was low. Students found writing papers, using Google Slides or PowerPoint to make presentations, and using textbooks among the least interesting online activities.

Figure 1

U.S. students' frequency and interest in online learning activities..

How did these tasks relate to students' academic engagement?

Table 2 shows the impact of each predictor on student academic engagement (high interest, skill, and challenge) in its own logistic regression model due to strong correlations in the frequency of activities assigned during remote learning. Each model controls for engagement at the beginning of the school year, race, gender. Students were clustered by school and classroom to account for variance that might occur due to school policy or teachers' familiarity with teaching online. When surveyed before the transition to remote learning, only 5 students reported academic engagement for every 100 who did not. The increase in academic engagement when measured in the fall showed the odds of reporting engagement were as high as 48 students for every 100 students who reported not being engaged. As shown in Table 2, when controlling for demographic data and their academic engagement (high interest, skill, and challenge) measure recorded in fall, there were strong significant correlations between academic engagement and the frequency of most of the project‐based activities related to the CESE intervention. The highest correlations to academic engagement were found in the frequency of equations modelling and participation in science and engineering practices, respectively, showing students to be 1.29 and 1.30 times more likely to report engagement. All types of modelling, except building models at home, were positively correlated with academic engagement, and students were between 1.17 and 1.30 times more likely to report engagement for each unit increase in frequency. Additionally, students who reported more frequent opportunities to take pride in their science achievements were 1.26 times more likely to report engagement with each unit increase in frequency. Many of the activities specifically related to remote teaching were not significantly correlated with academic engagement, and the highest correlations again corresponded with more project‐based tasks. For example, the odds of a student reporting engagement were 1.18 times higher with more frequent at home experiments during remote learning and 1.20 times higher with increase in frequency of building presentations with Slides or Power Point. Despite listing textbook use as uninteresting, students were 1.19 times more likely to report being engaged with each increase in frequency of reported use. Building models at home is not shown in this table because it was not significantly correlated to academic engagement and the logit model did not converge when controlling for other factors.

TABLE 2

Logistic regression coefficients showing the impact of each activity on academic engagement

	Correlation coefficient		Logit regression coefficient (β)		SE (β)	Odds ratio (eβ)
Items specifically related to PBL
Ask questions about phenomena	0.12	**	0.20	*	0.09	1.23
Build models	0.09	*	0.15		0.08	1.16
Class discussions about phenomena	0.13	***	0.22	**	0.08	1.25
Computer modelling 1	0.13	***	0.20	**	0.07	1.22
Computer modelling 2	0.11	**	0.15	*	0.06	1.17
Draw visual models	0.10	*	0.16	**	0.05	1.18
Equations modelling	0.15	***	0.27	**	0.10	1.30
Opportunities to ask questions	0.05		0.11		0.12	1.11
Opportunities to take pride	0.14	***	0.23	**	0.08	1.26
Present their findings	0.13	***	0.21	**	0.07	1.23
Science and engineering practices	0.15	***	0.26	***	0.07	1.29
Work together to understand phenomena	0.11	**	0.17	**	0.06	1.18
Items specifically related to remote teaching
Discussion board	0.06		0.08		0.05	1.09
Experiments to try at home	0.10	**	0.17	**	0.06	1.18
Live lessons	0.08	*	0.11		0.06	1.12
One‐on‐one video chat with teacher	0.10	**	0.15	**	0.05	1.16
Online simulations	0.07		0.13		0.07	1.14
Presentations using slides or power‐point	0.10	**	0.18	*	0.07	1.20
Recorded lessons	0.02		0.02		0.04	1.02
Text‐based instructions	0.08	*	0.13		0.08	1.13
Textbook use	0.10	*	0.18	**	0.05	1.19
Watching videos of experiments	0.07		0.14		0.08	1.15
Working in groups through video chat	0.09	*	0.14		0.07	1.15
Writing papers	0.09	*	0.15	**	0.04	1.16

Note. Due to the high correlations between activities, each activity was run in its own logistic regression model. The coefficients represent the impact of the activity on engagement when controlling for prior engagement, race, gender, and variance at the school and classroom levels.

p < 0.05.

p < 0.01.

p < 0.001.

How did engagement during the pandemic impact students' future aspirations?

In order to understand students' persistence during the pandemic, we explored the changes students made to their educational plans by comparing their responses in spring to those from the beginning of the year using the binary college indicator. Prior to the shift to remote instruction, 68.66% of students planned to attend four or more years of college. When measured again during remote learning, the number of students planning to attend college or graduate school increased significantly (p < 0.05), with the odds of a student reporting plans to attend college or graduate school rising from 2.19 to 2.6. Because the GPAs of students who reported they “do not know” were more similar to students planning 2 to 4 years of college than those who planned to attend trade school or no post‐secondary education, we anticipated that much of this change would come from students affirming their plans for college. When omitting students who reported they did not know their plans on either the background or exit surveys, there were no significant differences from the beginning to the end of the year. Additionally, fewer students reported not knowing their plans during the pandemic than when surveyed at the beginning of the year (p < 0.001).

To see how academic engagement impacted our measure of student persistence, we next used a two‐level logistic regression, again accounting for variance between classrooms, with the binary college ambition indicator as the outcome. As shown in Table 3, when controlling for race and gender, we found that both GPA and academic engagement had significant correlations to plans to attend four or more years of college during the pandemic, even when controlling for previous ambitions. Students who reported being engaged in their science courses during the pandemic were 2.19 times more likely to report plans to attend college or graduate school. The odds of reporting plans to attend college or graduate school were also 1.8 times higher for each unit increase in grade point average. Teacher level random effects were non‐negligible.

TABLE 3

U.S. students' plans to attend four or more years of college and engagement

	β	SE β	OR (eβ)
Previous plans to attend four or more years of college	3.56 ***	0.56	35.01
Female (male comparison)	−0.70	0.56	0.50
Race (White non‐Hispanic comparison)
Hispanic	−0.41	0.64	0.66
Black	0.69	0.72	1.99
Other a	−3.00 *	1.40	0.05
Asian b	0	0	1.00
Multiple	0.78	0.59	2.18
GPA	0.59 *	0.24	1.80
Academic engagement during pandemic	0.78 **	0.30	2.19
Teacher level random effects	0.10	0.45

Note. The analytic sample for this table is students who reported their educational ambitions both before and during the pandemic.

p < 0.05.

p < 0.01.

p < 0.001.

Only three students in the final analytic sample listed their race as Other, one of those three students selected a lower level of education.

All students who listed their race as Asian reported plans to attend four or more years of college both before and during the pandemic.

Finnish results

How did Finnish students engage situationally during remote teaching?

Nearly half of students indicated interest in their science activities (44.5%); however, only one quarter (29%) of experiences were identified as leaving students feeling skilled and more than one third (34.2%) were identified as challenging (see Table 4). When OLMs were calculated from these three measures, only 4.7% of science experiences were engaging moments. The mean for academic engagement was only 0.05 (min 0 − max 1).

TABLE 4

Finnish students' OLM situational engagement during pandemic

	N	M	SD	Percentageof occurrence (%) a
Interest	701	2.49	0.71	44.5
Skill	696	2.18	0.73	29.0
Challenge	700	2.31	0.71	34.2
OLM, situational engagement b	701	0.05	0.21	4.7

Occurrence is defined as choosing 3 (much) or 4 (very much) in the scale.

Situational engagement is defined as the joint occurrence of interest, skill, and challenge.

What kinds of activities were Finnish students participating in during remote teaching?

To understand the learning activities students participated in while learning remotely, we first summarised the mean level of each of the learning activities that have been reported in the general survey (see Figure 2). Differences in frequency were examined using a one‐way analysis of variance [ANOVA] (F = 158.5, df = 22, p < 0.001). The most often mentioned activities were following teacher's instruction, following demonstrations, and doing independent assignments. The least mentioned activities were doing tests, sharing documents, presenting, making a video or animation, and doing experiments at home. Discussion and interaction with peers and teachers (e.g. online chat, helping each other) were mentioned at the moderate level. Studying from a website and viewing videos and animations were common activities during the pandemic, though their frequencies were lower than teachers' direct instruction and independent work.

Figure 2

Frequency of learning activities from general survey..

Students' real‐time situational learning activities were also compared using chi‐square tests. When pooling the data, significant differences were found among these activities (χ2 = 4085.6, df = 17, p < 0.001). Following teachers' instruction, doing tasks or assignments independently, studying from books and solving mathematical problems were more represented than other activities.

Real‐time situational learning activities were then divided into three groups based on their reported frequency in the 701 situational responses: activities that occurred more than 50% of the time; those that occurred from 49% to 11% of the time; and those that occurred less than 10% of the time. Using cross‐tabulation analysis, we found that more frequently employed activities, such as following teachers' instruction, which happened over 50% of the time, were less successful in facilitating academic engagement (adj. residual = −2.47) than those activities that were classified as medium or low frequency (see Table 5). We then compared the level of interest, skill, and challenge across activities using one‐way ANOVA (see Table 6). There were significant differences across activities for interest and skill but not challenge. Post‐hoc analyses again confirmed that students were less interested and felt less skilled in activities that occurred the most frequently. In other words, the most common activities students experienced were the least engaging.

TABLE 5

Cross‐tabulation analysis of situational engagement per ESM activity group

		Situational engagement
Activity group		Not occurred	Occurred	Total
High frequency	Count	1365	75	1440
	Std residual	0.42	−1.61
	Adj std residual	2.47	−2.47
Medium frequency	Count	972	77	1049
	Std residual	−0.36	1.38
	Adj std residual	−1.83	1.83
Low frequency	Count	143	14	157
	Std residual	−0.34	1.32
	Adj std residual	−1.41	1.41
Total		2480	166	2646

Note. High frequency activities include following teachers' instruction, doing tasks independently, and book studying; medium frequency activities include solving math problems, writing, studying from a website, discussing online, constructing an explanation, using a model, analysing data, evaluating information, and asking questions; low frequency activities include using evidence to make an argument, developing a model, conducting an investigation, planning an investigation, making videos, and other.

TABLE 6

ANOVA results for interest, skill, and challenge per ESM activity groups

	High frequency activities	Medium frequency activities	Low frequency activities	F	Post‐hoc
Interest	2.54	2.76	2.76	30.66, df = 2, p < .001	High < medium, low
Skill	2.23	2.37	2.39	11.58, df = 2, p < .001	High < medium, low
Challenge	2.31	2.33	2.38	0.76, df = 2, p = .47	ns

Note. High frequency activities includes following teachers' instruction, doing task independently, and book studying; medium frequency activities includes solving math problem, writing, studying from website, discussing online, constructing an explanation, using a model, analysing data, evaluating information, and asking questions; low frequency activities includes using evidence to make an argument, developing a model, conducting an investigation, planning an investigation, making videos, and other.

What did the social and emotional learning of the students look like while learning remotely in Finland?

Among the six types of social and emotional learning experiences, when measured situationally the most salient was the importance of learning for the future (see Table 7). Close to half of responses (45.5%) indicated students felt that what they were learning was useful for their future. The likelihood of reporting being confident (30.44%), curious (28.1%), or persistent (i.e. gritty; 27.5%) was modest. Correlation analyses show that when students felt their learning was important for their future, and were moderately curious, persistent, and confident about themselves, they were more likely to be situationally engaged (OLM).

TABLE 7

Situational engagement (optimal learning moments) and social emotional learning

	M	SD	Occurrence (%) a	1	2	3	4	5	6
1. Situational engagement	0.05	0.21	4.71
2. Future importance	2.53	0.79	45.56	0.15 **
3. Lonely	1.38	0.72	8.73	−0.08 *	0.12 **
4. Bored	1.80	0.82	17.74	−0.17 **	−0.13 **	0.30 **
5. Confident	2.18	0.83	30.39	0.14 **	0.22 **	−0.14 **	−0.19 **
6. Curious	2.09	0.87	28.10	0.12 **	0.30 **	0.06	−0.26 **	0.30 **
7. Grit	2.12	0.82	27.48	0.12 **	0.27 **	0.06	−0.24 **	0.42 **	0.54 **

* p < 0.05.

p < 0.01.

Occurrence is defined as choosing 3 (much) or 4 (very much) in the scale.

DISCUSSION

When measured during the pandemic, U.S. students reported greater interest and challenge in their science subject than they did in Fall 2019. Generally, academic engagement for U.S. students showed an increase, but Finnish results showed that situational academic engagement was low. While it is impossible to distinguish a causal relationship, it is possible that this difference in results suggests students are less engaged in the specific activities they do remotely but were influenced by factors outside the classroom that increased overall engagement. For example, one or more components of engagement (interest, skill or challenge) increased for students because of what they were seeing in the news regarding the novel coronavirus.

Academic engagement was positively correlated with a number of project‐based activities employed during the pandemic. Students showed a significant increase in the odds of reporting engagement with more frequent use of science and engineering practices, class discussions, working together to understand phenomena, various modelling activities, presenting their work, and conducting experiments at home. Textbook use remained high among students who reported an overall sense of engagement. If teachers are using the textbooks for homework, this could be related to previous findings that students show above average situational engagement while doing math problems (Inkinen et al., 2020). Consistent with findings from Domina et al. (2021) more frequent opportunities for social and emotional learning (e.g. taking pride in science achievements and working in groups with peers) were also positively correlated with academic engagement. The Finnish study found that the less frequently assigned active practices (e.g. asking questions, analysing data) were better than passive activities in facilitating interest and skill. Similarly, there was a correlation between students reporting that they were engaged and the frequency of real science and engineering practices in their U.S. classrooms.

Unfortunately, the shift to remote learning did not allow students to engage in the same sorts of hands‐on, project‐based activities they would in a normal CESE classroom. For Finnish students, the most frequent learning activities during the pandemic were following teacher's instruction or demonstration and doing independent tasks or assignments. In the United States, students reported high interest but few opportunities to try experiments at home. This was also the least frequently reported activity for students in Finland, which may be due to the difficulty associated with conducting science investigations without experimental tools and support from teacher and peers. Additionally, safety concerns and difficulty with distributing or obtaining resources have been shown to hinder participation in experimental activities while learning science remotely (Kelley, 2020). Students in both countries reported few opportunities to collaborate with one another in trying new activities and problem solving while learning remotely.

Analysis of specific challenges faced by students in Finland reveals that many students had difficulties in planning their studies while learning remotely. Compared to the challenge of study planning, students had fewer challenges regarding technical problems or a place to study at home. Students may be unfamiliar with effective time management practices when leaving the structured environment of their in‐person high school classes. Finding ways to help students in planning their multiple assignments may benefit students who are learning in this less structured remote learning environment.

Despite facing numerous difficulties and challenges, there were some positive findings regarding students' social and emotional experiences. Despite reporting low situational engagement, nearly half of the Finnish students surveyed still felt that what they were learning in their science classes was important to their future. In the United States, education aspirations remained high and a significant proportion of the surveyed students raised or affirmed their ambitions toward college. This effect may be driven by students who previously did not know their plans deciding to attend college. For students who reported their future academic plans before and after the pandemic, academic engagement was significantly correlated with plans to attend college or graduate school. Regardless of the challenges the U.S. students faced and the difficulties they experienced with the organisation and management of the transition to receiving lessons remotely, students remained positive about their future education. If we consider this ambition a testament to persistence, there was a significant correlation between academic engagement and persistence in both countries.

Limitations

In the U.S. sample, the most at‐risk students were often unable to participate in the remote learning experience. The U.S. sample of students participating in remote learning was skewed toward students who had access to computers in districts that supported remote learning. Due to socio‐economic barriers, students in the United States were not graded and could not be held accountable for attendance. This lack of incentive may imply that those students who continued to participate shared certain characteristics. While socioeconomic disparities did not impact students' participation in Finland, this study did not follow the same students in a longitudinal study; the results are mainly framed by participation in full remote days of instruction.

In both studies, academic engagement was comprised of three components, each represented by only one question. The three questions are not expected to measure the same construct, but instead to indicate when three independent concepts occur together, which is reflected in the low Cronbach's alpha (0.61). Moving forward, measuring each construct with a multiple question scale could provide more reliable results.

Implications

It is important to emphasise that in both the United States and Finland active pedagogical practices in distance‐learning environments are found to be the most engaging among these students. By acknowledging the difficulties to conduct these proactive practices remotely, schools should provide support and sufficient tools to promote effective science learning. Although Finland's unified government experienced a somewhat smoother transition to remote learning than the United States, both countries had their own set of student challenges. In both countries, students lacked opportunities to engage in scientific practices used by real scientists in the field.

Although improving engagement and promoting positive social and emotional learning is undoubtedly a challenge for remote instruction, it is one that needs attention regardless of how soon the pandemic ends. Remote or hybrid learning situations are likely to continue for the long‐term. The activities students were most interested in involved doing science rather than simply reading about it, which is what the CESE intervention emphasises. In the full sample in the prior year, a positive effect for science learning was found among treatment students, including for low‐income and minority students who were over‐sampled (Schneider et al., under review). This leads to concerns of potential learning loss when students are unable to participate in experiential science learning. Collaboration and experimentation are key practices used by real scientists working in the field. To optimise students' science learning, remote classroom environments may use and adapt existing technologies that allow students to remain engaged with their lessons with opportunities to figure out phenomena. This engagement with science may in turn encourage students toward more ambitious educational goals.

CONCLUSION

Similar to results from previous years of the CESE intervention, during remote learning in the 2019–2020 school year, students in the CESE study showed that engagement was strongly correlated with a variety of project‐based activities assigned by their teachers. In both the United States and Finland, students reported higher engagement with some of the least frequently assigned activities. When learning remotely, students show more engagement when performing real science and engineering practices like conducting investigations at home, performing modelling activities, asking questions, and participating in class discussions. This engagement is in turn related to positive social and emotional outcomes such as confidence, persistence, and ambition. The likelihood of engagement is much lower in the remote learning environments than in the normal classroom setting. Consequently, attention should be paid to providing equitable opportunities to participate in project‐based learning activities whether learning remotely or in a classroom.