Student and School Characteristics Associated With COVID-19-Related Learning Decline Among Middle and High School Students in K-12 Schools.

BACKGROUND: COVID-19-disrupted schools, including shifts to virtual learning which may have impacted academic progress. This study assessed characteristics associated with changes in academic grades (before and during the pandemic) for different learning modalities for US students ages 13-19.
METHODS: Students (N = 2152) completed a web survey on school-related experiences during the 2020-2021 school year. County social vulnerability and SARS-CoV-2 transmission data were merged with survey data. Multivariable logistic regression analysis for grade change was conducted with student and school characteristics for each learning modality, controlling for community characteristics.
RESULTS: Greater proportions of remote/virtual (34.4%) and hybrid (30.1%) learning students reported grade decline compared to in-person students (19.9%). Among in-person students, odds of reporting same/improved grades were 65% lower among non-Hispanic black students and 66% lower among non-Hispanic students from other races, compared to non-Hispanic white students. Among hybrid students, odds of reporting same/improved grades for students reporting anxiety were 47% lower than students without anxiety, and odds of reporting same/improved grades among students reporting substance use were 40% lower than students not reporting substance use. Among remote/virtual students, odds of reporting same/improved grades among students with depression were 62% lower than odds of students not reporting depression symptoms. Remote/virtual students who received school-provided educational services also had 1.55 times the odds of reporting same/improved grades, compared to remote/virtual students not receiving these services.
CONCLUSIONS: Academic grades were negatively impacted during COVID-19 and learning mode may have contributed. Understanding these impacts is critical to student health and academic achievement. Published 2022. This article is a U.S. Government work and is in the public domain in the USA.

In March 2020, the COVID‐19 pandemic led to an unprecedented change to traditional learning in US schools. Nearly, 93% of all students in the US engaged in some form of virtual learning in Spring 2020 with many school closures continuing into the following year. The extent to which student learning modality (eg, attended school in‐person or virtually) affected academic progress during the pandemic is unknown. To better understand how academic progress may have been affected by the pandemic, it is useful to consider the role of learning modality as well as other student, school, and community characteristics.

The Social Ecological Model (SEM) provides a framework to understand how a student's individual characteristics (eg, mental health, academic ability) interact with social relationships (eg, peers, school, family) and environmental factors (eg, neighborhood social capital) to influence health outcomes. , , Academic achievement has also been examined through the SEM lens. The connections between individual, family, and school characteristics and academic achievement, including standardized test scores, are well documented , , , , , , and form the foundation for the Whole School, Whole Community, Whole Child model. This model illustrates that multiple components and systems influence a school's ability to ensure students are safe, supported, challenged, engaged, and healthy. ,

There is evidence that overall student achievement, regardless of learning modality, was impacted by the pandemic. For example, in a survey of school district administrators during the 2020‐2021 school year, 34% reported a substantial increase in the percentage of high school students receiving poor grades, with greater increases in districts serving mostly black and Hispanic students or districts with historically lower academic achievement levels. Signs of learning loss and disparities are also evident in standardized test scores—one study reported drops in reading and math scores, compared to prepandemic scores of students in the same grade level (with greatest declines among black and Hispanic youth). During COVID‐19, it is possible that student learning modality and community characteristics (eg, level of social vulnerability and community social capital ) may have interacted, contributing to or exacerbating learning disparities among US K‐12 students. For example, socially vulnerable counties , experienced higher COVID‐19 case counts and fatality rates and, as a result, may have been more likely to mandate virtual‐only instruction. Given disparities in full‐time, in‐person learning by race/ethnicity have been demonstrated during the pandemic across school levels and by geographic region and state, learning modality might be a potential mechanism that contributed to grade declines.

Learning modality might have also influenced students' sense of belonging at school, emotional wellbeing, and access to school‐provided supports. Emerging research indicates that students in virtual‐only instruction were less likely to report feeling connected to school and urban and large districts were less likely to provide 100% in‐person instruction than rural, suburban, and smaller jurisdictions. These findings suggest potential disparities in school connectedness and academic achievement, dependent on school instruction mode and location. Further, a myriad of negative effects on adolescent mental health and well‐being related to COVID‐19 have been documented, suggesting a mental health crisis among adolescents, many of whom were already experiencing challenges. These challenges may have been further exacerbated by school closures and virtual learning as schools provide important mental health services and learning supports. Schools are one of the primary providers of mental health services to students ; however, the extent to which schools provided such services during the pandemic is unclear. , School connectedness and access to health‐related supports and services can buffer the negative impact of individual risk factors on adolescent wellbeing and improve overall mental health , and, consequently, academic achievement. , Finally, service gaps might disproportionately impact students from low‐income households and racial/ethnic minority groups who are more likely than other groups to rely solely on educational settings for mental health services.

The purpose of this study is to use SEM principles to describe self‐reported grade change during the pandemic, and to identify individual, school, and community characteristics associated with grade change, within each of 3 learning modalities.

METHODS

Study Design and Participants

Data were collected as part of the Monitoring School COVID‐19 Mitigation Strategies Project (a CDC Foundation project, funded by donors to its' COVID‐19 Emergency Response Fund, to inform CDC's Operational Strategy for K‐12 Schools through Phased Prevention COVID‐19 recommendations). , Survey data on student experiences during the 2020‐2021 school year were collected May 10‐June 1, 2021, using Qualtrics (Provo, UT), an online survey platform that provides access to specific populations through double‐opt‐in market research panels. Qualtrics samples are intended to be nationally representative based on US Census estimates and include quotas for demographic characteristics.

Web panel members who identified as a parent of a teen aged 13 or older in middle or high school were asked to provide consent for their child to participate. If consent was provided, a survey invitation was emailed to the teen. Inclusion criteria were students ages 13 and above; attending a US‐based public, private or charter middle or high school; and provided assent to participate. Among the 7589 respondents initiating the survey, 5231 were screened out because they did not meet criteria, they did not finish the survey, or quotas had already been met. An additional 183 surveys were excluded for data quality issues, resulting in 2175 respondents. Twenty‐three additional surveys were dropped for students who indicated they were “transgender,” “non‐binary,” or “non‐conforming”—the small sample would not allow for reliable estimates for this population. The final analytic sample was 2152 respondents (Figure 1).

Figure 1

Study Inclusion Criteria for Final Analytic Sample

CDC/ATSDR Social Vulnerability Index (SVI) and SARS‐CoV‐2 transmission data were obtained from publicly available data based on the counties where students reported they attended school during the 2020‐2021 school year. The SVI ranks US Census tracts and counties on 15 social factors, with higher scores denoting higher vulnerability. County‐specific data from 2018 SVI data set (most recent available) were analyzed. County‐specific SARS‐CoV‐2 transmission levels (based on new positive case rate and percent positivity) were obtained from the CDC COVID Data Tracker to characterize disease burden at time of data collection.

Variables

Change in academic grades (primary outcome) was constructed from 2 survey questions: “How would you describe your grades in school now?” and “How would you describe your grades in school before the COVID‐19 pandemic (before February 2020)?” Response options for each question were “mostly A's,” “mostly B's,” “mostly C's,” “mostly D's,” “mostly F's,” “none of these grades,” or “not sure.” A dichotomous variable was created to categorize grades as “declined” or “same/improved” since the pandemic began for respondents who provided letter grade information for both questions (excluding those who reported “none of these grades” or “not sure” for one or both questions). To assess learning modality, respondents were asked to characterize how they attended school most often during the 2020‐2021 school year: “in‐person 100% (coming into school),” “remote/virtual learning (staying at home) 100%,” or “hybrid (a mix of coming into school and staying at home).”

Student characteristics

Sociodemographic characteristics included gender (female, male), race/ethnicity (Hispanic, non‐Hispanic white, non‐Hispanic black, non‐Hispanic other), and school level (middle school—grades 6‐8, high school—grades 9‐12). The non‐Hispanic other race/ethnicity category includes respondents who selected “Asian,” “American Indian or Alaska Native,” “Native Hawaiian or Pacific Islander,” “some other race,” or multiple race/ethnicity categories on the survey. The non‐Hispanic other category is not disaggregated due to small cell size generating unstable estimates and, also, to protect confidentiality.

Health and behavioral characteristics were derived from mental health and substance use variables. The Patient Health Questionnaire 2 (PHQ‐2) and Generalized Anxiety Disorder (GAD‐2) were used to assess depression and anxiety, respectively. These scales are abbreviated versions of the PHQ‐9 and GAD‐7 scales, respectively, which have been validated in adult primary care and extended to general samples of adolescents , , and young adults. , PHQ‐2 and GAD‐2 have been demonstrated to perform similarly to the longer scales with regard to identifying young people with probable mood or anxiety disorders. Respondents were asked, “Over the last 2 weeks, how often have you been bothered by the following problems?” Anxiety items were: “feeling nervous, anxious, or on edge” and “not being able to stop or control worrying.” Depression items were: “little interest or pleasure in doing things” and “feeling down, depressed, or hopeless.” Response choices for each of these 4 survey items were “never,” “rarely,” “sometimes,” “often,” or “very often.” Response choices “rarely” and “sometimes” were combined for each survey item and then the following scores were assigned (never = 0, rarely/sometimes = 1, often = 2, very often = 3). Using the GAD‐2 scale, respondents with anxiety symptoms represent those who scored ≥3 after summing both anxiety items (range = 0‐6). Using the PHQ‐2 scale, respondents with depression symptoms represented those who scored ≥3 after summing both depression items (range = 0‐6). Substance use was assessed by asking respondents, “Since the pandemic began (February 2020), have you started using or increased using substances to help you cope with stress or emotions (substance use includes tobacco, alcohol, legal or illegal drugs, or prescription drugs that are taken in a way not recommended by your doctor)?” Survey respondents responded “yes” or “no” for each of the following substances: alcohol, non‐prescription drugs, prescription drugs, tobacco products. A dichotomous variable was created to categorize substance use as any use (ie, responded “yes” to at least 1 substance) or no use (ie, responded “no” to all). Although this substance use measure was not validated prior to use in this study, similar retrospective substance use measures have been reported in other research.

School‐provided supports

We focused on student‐reported receipt of services to support learning and health to assess whether schools were providing these services to students as needed. Survey respondents indicated the frequency with which they received education‐related services during the 2020‐2021 school year: in‐school or after school tutoring, accommodations for assignments (eg, teacher gave them more time to complete assignments, provided additional instruction after class), academic support (changes to classroom assignments or extra help from the teacher), and individual classroom support (a second teacher or aide helped them in the classroom). Respondents also indicated how often they received health‐related services: occupational therapy, speech therapy, and behavioral/mental health support (by school counselor, classroom teacher, or other school staff member). For each service type, a separate dichotomous variable was created to categorize responses as either received any service (ie, responded “yes” to at least 1 service) or no services (ie, responded “no” to all).

Community characteristics

US Census region (Midwest, Northeast, South, West) and locality (rural, suburban, urban) of the respondent's school were obtained through the survey. Tertiles were computed to identify counties in the bottom one‐third of the SVI range (low vulnerability), middle one‐third (moderate vulnerability), and top one‐third (high vulnerability). SARS‐CoV‐2 transmission level was calculated as the number of days during the survey data collection period that the county where the school is located met criteria for high transmission level. Transmission data were available for 15 of the 22 data collection days.

Data Analysis

Analysis was conducted with Stata Version 14 (College Station, TX). Survey data were weighted to be representative of the sampled population using demographic variables (age, gender, race/ethnicity). A multipurpose Iterative Proportional Fitting procedure was used to calibrate individual‐level weights, simultaneously adjusting for population estimates from 2019 National Health Interview Survey ; bloc‐level nonresponse adjustment based on calibration in the quintiles of estimated propensity to respond to surveys; and weight trimming procedures (removed 5% of extreme high/low estimates).

Pearson χ2 tests were conducted to identify unadjusted, bivariate associations between learning modality and student characteristics, school‐provided supports, and community characteristics (Table 1) to illustrate the prevalence and relationship between these variables before subpopulating the analyses for logistic regression. Bivariate analyses were also conducted between academic grade change (outcome measure) and student characteristics and school‐provided supports (Table 2) to 1) illustrate the prevalence and relationship between the variables before subpopulating the analyses for logistic regression, and 2) inform which variables should be included in the models, a part of model specification. In Table 2, we focused on student and school characteristics which may be most useful to school decision makers looking to identify students who might need more academic support.

Table 1

Characteristics of Students, School‐Provided Supports, and Community Characteristics of Survey Respondents Ages 13‐19 Attending K‐12 Schools by Learning Modality (In‐Person, Hybrid, or Remote/Virtual) for the SY 2020‐2021

			Student Learning Modality, N, % (95% CI)
	Overall, N, % (95% CI)		In‐person		Hybrid		Remote/Virtual		p‐Value
Total	2152		425 (19.8%)		884 (41.1%)		843 (39.2%)
Academic grade change*									<.001
Declined	632	29.8 (27.8, 31.9)	83	19.9 (16.2, 24.3)	267	30.1 (27.1, 33.4)	282	34.4 (31.1, 37.9)
Same/improved	1496	70.2 (68.1, 72.2)	338	80.1 (75.7, 83.8)	608	69.9 (66.6, 72.9)	550	65.6 (62.1, 68.9)
Student sociodemographic characteristics
Gender									.02
Female	1071	49.0 (46.7, 51.2)	205	46.2 (41.3, 51.2)	412	46.6 (43.2, 50.0)	454	52.8 (49.2, 56.3)
Male	1081	51.0 (48.8, 53.3)	220	53.8 (48.8, 58.7)	472	53.4 (50.0, 56.8)	389	47.2 (43.7, 50.8)
Race/ethnicity									<.001
Non‐Hispanic white	1117	52.8 (50.6, 55.0)	264	62.7 (57.8, 67.4)	527	61.3 (57.9, 64.6)	326	39.1 (35.7, 42.6)
Non‐Hispanic black	272	13.0 (11.6, 14.6)	44	11.4 (8.5, 15.2)	87	9.6 (7.8, 11.9)	141	17.2 (14.7, 20.1)
Hispanic	530	25.6 (23.7, 27.6)	83	19.9 (16.1, 24.2)	187	21.3 (18.6, 24.3)	260	32.9 (29.6, 36.3)
Non‐Hispanic other †	233	8.6 (7.6, 9.8)	34	6.0 (4.2, 8.4)	83	7.7 (6.2, 9.6)	116	10.8 (9.0, 13.0)
School level									.6
High school (grades 9‐12)	1367	67.7 (65.6, 69.6)	261	65.7 (60.9, 70.1)	559	67.7 (64.5, 70.7)	547	68.6 (65.4, 71.7)
Middle school (grades 6‐8)	785	32.3 (30.4, 34.4)	164	34.4 (29.9, 39.1)	325	32.3 (29.3, 35.5)	296	31.4 (28.3, 34.6)
Student health/behavioral characteristics
Mental health ‡
Has anxiety symptoms	478	22.3 (20.5, 24.2)	88	20.8 (17.0, 25.1)	215	24.1 (21.3, 27.2)	175	21.2 (18.4, 24.3)	.3
Has depression symptoms	422	19.5 (17.8, 21.3)	78	18.8 (15.1, 23.0)	174	19.0 (16.5, 21.9)	170	20.4 (17.7, 23.4)	.7
Substance use since pandemic began §	348	15.9 (14.3, 17.6)	79	19.2 (15.5, 23.5)	152	16.5 (14.2, 19.3)	117	13.5 (11.3, 16.1)	.04
School‐provided supports
Received education‐related services ‖	1515	70.1 (68.0, 72.1)	285	66.1 (61.2, 70.7)	633	71.9 (68.7, 74.8)	597	70.3 (67.0, 73.5)	.1
Received health‐related services ¶	288	13.1 (11.7, 14.7)	75	17.1 (13.7, 21.2)	111	12.8 (10.7, 15.4)	102	11.5 (9.4, 13.9)	.02
Community characteristics
US Census region #									<.001
Midwest	411	20.8 (19.0, 22.8)	96	25.0 (20.9, 29.7)	198	25.2 (22.2, 28.4)	117	14.3 (12.0, 17.0)
Northeast	385	16.5 (15.0, 18.1)	46	9.5 (7.1, 12.7)	190	19.9 (17.3, 22.6)	149	16.6 (14.2, 19.3)
South	873	38.6 (36.4, 40.7)	230	51.4 (46.4, 56.4)	314	33.2 (30.0, 36.4)	329	37.7 (34.3, 41.1)
West	483	24.1 (22.2, 26.1)	53	14.0 (10.7, 18.1)	182	21.9 (19.1, 24.9)	248	31.5 (28.2, 34.9)
Locality**									<.001
Rural	425	14.1 (12.8, 15.4)	106	18.5 (15.3, 22.3)	199	16.3 (14.2, 18.7)	120	9.5 (7.9, 11.4)
Suburban	1085	56.4 (54.2, 58.5)	220	58.5 (53.6, 63.2)	433	55.7 (52.2, 59.0)	432	56.0 (52.5, 59.4)
Urban	642	29.6 (27.6, 31.7)	99	23.0 (19.1, 27.4)	252	28.1 (25.1, 31.2)	291	34.5 (31.3, 37.9)
Social vulnerability index †† , ‡‡									<0.001
Low	750	35.7 (33.6, 37.9)	163	39.8 (35.0, 44.8)	331	38.9 (35.5, 42.3)	256	30.5 (27.3, 33.9)
Moderate	687	31.6 (29.6, 33.7)	138	33.2 (28.7, 38.1)	290	32.0 (28.9, 35.3)	259	30.5 (27.4, 33.9)
High	715	32.6 (30.6, 34.7)	124	27.0 (22.9, 31.6)	263	29.1 (26.1, 32.4)	328	39.0 (35.6, 42.5)
High days of SARS‐CoV‐2 transmission ‡‡ , §§									.05
0 days	1249	62.0 (59.8, 64.2)	254	64.0 (59.0, 68.7)	494	59.2 (55.6, 62.6)	501	64.0 (60.5, 67.4)
1‐7 days	472	22.3 (20.5, 24.3)	85	19.7 (16.0, 24.0)	200	22.9 (20.1, 26.0)	187	23.1 (20.1, 26.2)
8‐14 days	315	15.7 (14.1, 17.4)	187	16.4 (13.0, 20.5)	145	17.9 (15.3, 20.9)	102	12.9 (10.7, 15.6)

Table shows unweighted counts (N), weighted overall and column percentages (%), and weighted 95% confidence intervals (CIs). p‐Value represents associations between the groups of learning modalities by each variable of interest. p‐Values <.05 were considered statistically significant.

SY, school year.

Represents change in letter grades since before the start of the COVID‐19 pandemic (before February 2020). Grade change could not be computed for n = 24 survey respondents because they reported “none of these grades” or “not sure” for one or both questions about academic grades.

Non‐Hispanic other race/ethnicity category includes Asian, American Indian or Alaska Native, Native Hawaiian or Pacific Islander, some other race, or selected more than one race category. The Non‐Hispanic other category is not disaggregated due to small cell size generating unstable estimates and to protect confidentiality.

Using the GAD‐2 scale (Generalized Anxiety Disorder), respondents with anxiety symptoms represent those who scored ≥3 for anxiety, indicating a need for follow‐up screening for anxiety. Using the PHQ‐2 scale (Patient Health Questionnaire), respondents with depression symptoms represent those who scored ≥3 for depression, indicating a need for follow‐up screening for depression.

Started or increased use of at least 1 substance (alcohol, tobacco, prescription drug, or non‐prescription drug) to help cope with stress or emotions, taken in a way not recommended by a doctor.

Education‐related services received during SY 2020‐2021 (tutoring, accommodations for assignments, academic support, and/or individual classroom support).

Health‐related services received during SY 2020‐2021 (occupational therapy, speech therapy, and/or behavioral/mental health support).

Based on survey respondent's self‐reported state of residence.

Based on survey respondent's self‐reported location of school. This information was missing for n = 188 respondents.

2018 CDC/ATSDR Social Vulnerability Index (SVI) for the county where respondent school is located. Low = counties in bottom one‐third, or tertile, of SVI scores (lowest vulnerability). Moderate = counties in middle tertile of SVI scores. High = counties in top tertile of SVI scores (highest vulnerability).

Could not be computed for n = 188 survey respondents due to missing school location.

Number of days that the county where the school is located was designated as having a high SARS‐CoV‐2 transmission level (8‐14 days).

Table 2

Characteristics of Students and School‐Provided Supports of Survey Respondents Ages 13‐19 Attending K‐12 Schools by Academic Grade Change (Declined or Same/Improved) for the SY 2020‐2021

			Academic Grade Change*, N, % (95% CI)
	Overall, N, % (95% CI)		Declined		Same/Improved		p‐Value
Total		2128	632 (29.4%)		1496 (69.5%)
Student learning modality							<.001
In‐person	421	19.8 (18.1, 21.7)	83	13.3 (10.7, 16.3)	338	22.6 (20.5, 24.9)
Hybrid	875	40.6 (38.5, 42.8)	267	41.1 (37.2, 45.2)	608	40.4 (37.9, 43.1)
Remote/virtual	832	39.5 (37.4, 41.8)	282	45.6 (41.6, 49.7)	550	37.0 (34.4, 39.6)
Student sociodemographic characteristics
Gender							.4
Female	1059	48.9 (46.8, 51.2)	300	47.5 (43.4, 51.6)	759	49.6 (47.0, 52.3)
Male	1069	51.0 (48.8, 53.3)	332	52.5 (48.4,56.6)	737	50.4 (47.7, 53.1)
Race/ethnicity							.01
Non‐Hispanic white	1105	52.9 (50.7, 55.1)	294	48.4 (44.3, 52.5)	811	54.8 (52.1, 57.4)
Non‐Hispanic black	267	12.9 (11.4, 14.4)	90	14.2 (11.6, 17.3)	177	12.3 (10.6, 14.2)
Hispanic	525	25.7 (23.7, 27.7)	186	29.9 (26.3, 33.8)	339	23.8 (21.6, 26.2)
Non‐Hispanic other †	231	8.6 (7.6, 9.8)	62	7.4 (5.8, 9.6)	169	9.1 (7.8, 10.6)
School level							.8
High school (grades 9‐12)	1354	67.8 (65.7, 69.7)	400	67.5 (63.6, 71.0)	954	67.9 (65.5, 70.3)
Middle school (grades 6‐8)	774	32.2 (30.3, 34.3)	232	32.6 (29.0, 36.4)	542	32.1 (29.8, 34.5)
Student health/behavioral characteristics
Mental health ‡
Has anxiety symptoms	472	22.3 (20.5, 24.2)	196	31.7 (28.01, 35.7)	276	18.3 (16.3, 20.4)	<.001
Has depression symptoms	419	19.6 (17.9, 21.4)	183	29.5 (25.9, 33.4)	236	15.4 (13.6, 17.4)	<.001
Substance use since pandemic began §	344	15.9 (14.3, 17.6)	135	21.0 (17.8, 24.5)	209	13.7 (12.0, 15.6)	<.001
School‐provided supports
Received education‐related services ‖	1502	70.3 (68.2, 72.3)	436	68.5 (64.5, 72.2)	1066	71.0 (68.6, 73.4)	.3
Received health‐related services ¶	286	13.2 (11.8, 14.8)	80	11.9 (9.6, 14.7)	206	13.7 (12.0, 15.6)	.3

Table shows unweighted counts (N), weighted overall and column percentages (%), and weighted 95% confidence intervals (CIs). Academic grade change represents change in letter grades since before the start of the COVID‐19 pandemic (before February 2020). Grade change could not be computed for n = 24 survey respondents because they reported “none of these grades” or “not sure” for one or both questions about academic grades. p‐Values <0.05 considered statistically significant.

SY, school year.

Academic grade change represents change in letter grades since before the start of the COVID‐19 pandemic (before Feb. 2020). Grade change could not be computed for n = 24 survey respondents because they reported “none of these grades” or “not sure” for one or both questions about academic grades (overall sample size (2152) minus surveys without grade change information (24) = 2128).

Non‐Hispanic other race/ethnicity category includes Asian, American Indian or Alaska Native, Native Hawaiian or Pacific Islander, some other race, or selected more than one race category. The Non‐Hispanic other category is not disaggregated due to small cell size generating unstable estimates and to protect confidentiality.

Using the GAD‐2 scale (Generalized Anxiety Disorder), respondents with anxiety symptoms represent those who scored ≥3 for anxiety, indicating a need for follow‐up screening for anxiety. Using the PHQ‐2 scale (Patient Health Questionnaire), respondents with depression symptoms represent those who scored ≥3 for depression, indicating a need for follow‐up screening for depression.

Started or increased use of at least one substance (alcohol, tobacco, prescription drug, or non‐prescription drug) to help cope with stress or emotions, taken in a way not recommended by a doctor.

Education‐related services received during SY 2020‐2021 (tutoring, accommodations for assignments, academic support, and/or individual classroom support).

Health‐related services received during SY 2020‐2021 (occupational therapy, speech therapy, and/or behavioral/mental health support).

Multivariable logistic regression models were then conducted (Table 3). Regression analysis was first conducted with the entire sample. Then models were subpopulated by learning modality to assess the relationships between grade change and student and school characteristics for each learning group, controlling for community characteristics. Because county‐level SVI and SARS‐CoV‐2 transmission data were merged with survey data, we calculated the intraclass coefficient (ICC = 0.02) to determine any clustering effects. Because there were no clustering effects, we did not use hierarchical models.

Table 3

Multivariable Logistic Regression for Having Same or Improved Academic Grades (Compared to Prepandemic) and Characteristics of Students and School‐Provided Supports, Subpopulated by Student Learning Modality (In‐Person, Hybrid, or Remote/Virtual) in SY 2020‐2021

			Student Learning Modality
	Overall (N = 2012)		In‐person (N = 403)		Hybrid (N = 830)		Remote/Virtual (N = 779)
	OR (95% CI)	p‐Value	OR (95% CI)	p‐Value	OR (95% CI)	p‐Value	OR (95% CI)	p‐Value
Student learning modality			‐		‐		‐
In‐person	Ref.
Hybrid	0.58 (0.42, 0.80)	.001
Remote/virtual	0.50 (0.36, 0.69)	<.001
Student sociodemographic characteristics
Gender
Female	Ref.		Ref.		Ref.		Ref.
Male	0.85 (0.69, 1.05)	.132	1.04 (0.60, 1.80)	.902	0.74 (0.54, 1.03)	.071	0.90 (0.64, 1.26)	.537
Race/ethnicity
Non‐Hispanic white	Ref.		Ref.		Ref.		Ref.
Non‐Hispanic black	0.80 (0.57, 1.12)	.196	0.35 (0.15, 0.80)	.013	1.27 (0.72, 2.25)	.404	0.72 (0.43, 1.19)	.200
Hispanic	0.78 (0.60, 1.02)	.068	1.27 (0.61, 2.61)	.523	0.78 (0.51, 1.19)	.245	0.68 (0.45, 1.02)	.061
Non‐Hispanic other*	1.22 (0.83, 1.77)	.309	0.34 (0.13, 0.87)	.025	1.50 (0.81, 2.77)	.197	1.50 (0.85, 2.65)	.159
School level
High school (grades 9‐12)	Ref.		Ref.		Ref.		Ref.
Middle school (grades 6‐8)	0.91 (0.73, 1.12)	.368	1.19 (0.66, 2.14)	.560	0.82 (0.59, 1.14)	.240	0.96 (0.68, 1.36)	.825
Student health/behavioral characteristics
Mental health †
Has anxiety symptoms	0.60 (0.44, 0.81)	.001	0.67 (0.27, 1.7)	.396	0.53 (0.33, 0.84)	.007	0.66 (0.41, 1.06)	.082
Has depression symptoms	0.61 (0.44, 0.85)	.003	0.73 (0.27, 2.0)	.547	0.92 (0.54, 1.60)	.754	0.38 (0.23, 0.60)	<.001
Substance use since pandemic began ‡	0.71 (0.53, 0.95)	.022	0.60 (0.29, 1.2)	.163	0.60 (0.39, 0.92)	.021	0.84 (0.53, 1.34)	.459
School‐provided supports
Received education‐related services §	1.31 (1.04, 1.65)	.024	0.91 (0.48, 1.74)	.783	1.33 (0.93, 1.92)	.120	1.55 (1.08, 2.22)	.018
Received health‐related services ‖	1.36 (0.98, 1.89)	.069	1.02 (0.44, 2.33)	.967	1.28 (0.77, 2.10)	.340	1.47 (0.84, 2.56)	.174

Overall model was not subpopulated by learning mode. All models controlled for community characteristics (Census region, locality, Social Vulnerability Index, and SARS‐CoV‐2 transmission level). Significant findings (p < .05) are bolded. Referent group for each mental health measure is “no” (did not report symptoms at level that warranted additional screening). Referent group for the outcome variable, academic grade change, is declined. Referent group for substance use is “no” (did not report any substance use). Referent group for each school‐provided support measure is “no” (did not receive any services of that type).

CI, confidence interval; OR, odds ratio; SY, school year.

Non‐Hispanic other race/ethnicity category includes Asian, American Indian or Alaska Native, Native Hawaiian or Pacific Islander, some other race, or selected more than 1 race category. The Non‐Hispanic other category is not disaggregated due to small cell size generating unstable estimates and to protect confidentiality.

Using the GAD‐2 scale (Generalized Anxiety Disorder), respondents with anxiety symptoms represent those who scored ≥3 for anxiety, indicating a need for follow‐up screening for anxiety. Using the PHQ‐2 scale (Patient Health Questionnaire), respondents with depression symptoms represent those who scored ≥3 for depression, indicating a need for follow‐up screening for depression.

Started or increased use of at least 1 substance (alcohol, tobacco, prescription drug, or non‐prescription drug) to help cope with stress or emotions, taken in a way not recommended by a doctor.

Education‐related services received during SY 2020‐2021 (tutoring, accommodations for assignments, academic support, and/or individual classroom support).

Health‐related services received during SY 2020‐2021 (occupational therapy, speech therapy, and/or behavioral/mental health support).

All findings were considered statistically significant if p‐value <.05.

RESULTS

Table 1 depicts descriptive characteristics of respondents for overall sample and by learning modality (unweighted counts and weighted percentages), including corresponding bivariate associations. Learning modality was significantly associated with academic grade change, gender, race/ethnicity, substance use, receiving school‐provided health‐related services, Census region, locality, and SVI. Among in‐person students, about one‐fifth (19.9%) reported their grades declined compared to prepandemic while 34.4% of remote/virtual students reported their grades declined. Non‐Hispanic white students represented 62.7% of in‐person students but only 39.1% of remote/virtual students. For all other race/ethnic groups, there were greater proportions of students from those groups attending school remotely/virtually, compared to in‐person. Greater proportions of in‐person (19.2%) and hybrid (16.5%) students reported substance use, compared to remote/virtual students (13.5%). Additionally, while 17.1% of in‐person students reported they received health‐related services from school, only 12.8% of hybrid students and 11.5% of remote/virtual students received these services. Students from the South represented the largest group in each learning modality and the distribution of different localities (rural, suburban, urban) was similar for all modalities. Overall, about a third of students (32.6%) were from high SVI counties and county SVI varied across learning modality. In‐person students were primarily from low SVI counties (39.8%) and remote/virtual students were primarily from high SVI counties (39.0%).

Table 2 depicts prevalence estimates of student characteristics and school‐provided supports and academic grade change (unweighted counts and weighted percentages), including corresponding bivariate associations. Learning modality, race/ethnicity, anxiety symptoms, depression symptoms, and substance use were significantly associated with grade change. Among students whose grades declined, 45.6% were remote/virtual students, 41.1% were hybrid students, and 13.3% were in‐person students. With regard to race/ethnicity, non‐Hispanic white students comprised 54.8% of students whose grades stayed the same or improved but only 48.4% of students whose grades declined. The opposite pattern was true for non‐Hispanic black and Hispanic students—there were greater proportions of non‐Hispanic black (14.2%) and Hispanic (29.9%) students in the declined grades group, compared to the same/improved grades group (non‐Hispanic black = 12.3%; Hispanic = 23.8%). For each mental health measure, a greater proportion of students in the group that reported their grades declined reported experiencing anxiety or depression symptoms, compared to those in the same/improved grades group. Finally, 21% of the declined grades group reported substance use compared to only 13.7% of the same/improved grades group.

Results from the multivariable logistic regression models, subpopulated by learning modality, are shown in Table 3. Controlling for community characteristics, significant associations with academic grade change were observed for student characteristics and school‐provided supports. Associations varied depending on learning modality. Regression results for each learning modality are described below. Prevalence estimates and regression results for the community characteristics are provided in Appendix Tables A1 and A2.

Table A1

Community Characteristics Corresponding to the Geographic Location of Students Ages 13‐19 Attending K‐12 Schools and Academic Grade Change (Declined or Same/Improved) in SY 2020‐2021

			Academic Grade Change*, N, % (95% CI)
	Overall, N, % (95% CI)		Declined		Same/Improved		p‐Value
Total	2128		632 (29.4%)		1496 (69.5%)
Community characteristics
Census region †							.14
Midwest	406	20.8 (19.0, 22.8)	130	22.2 (18.9, 25.8)	276	20.3 (18.1, 22.6)
Northeast	381	16.5 (15.0, 18.1)	116	17.1 (14.3, 20.2)	265	16.3 (14.5, 18.2)
South	863	38.5 (36.4, 40.7)	231	34.6 (30.9, 38.5)	632	40.2 (37.6, 42.8)
West	478	24.2 (22.3, 26.2)	155	26.2 (22.7, 30.0)	323	23.3 (21.1, 25.7)
Locality ‡							.95
Rural	421	14.1 (12.8, 15.5)	121	13.1 (10.9, 15.6)	300	14.5 (13.0, 16.3)
Suburban	1075	56.4 (54.2, 58.6)	303	54.3 (50.2, 58.3)	772	57.4 (54.7, 60.0)
Urban	632	29.5 (27.5, 31.5)	208	32.7 (29.0, 36.6)	424	28.1 (25.8, 30.6)
Social vulnerability index § , ‖							.019
Low	745	35.9 (33.8, 38.1)	193	31.2 (27.5, 35.1)	552	37.9 (35.3, 40.5)
Moderate	678	31.6 (29.6, 33.7)	213	33.8 (30.0, 37.7)	465	30.7 (28.3, 33.2)
High	705	32.5 (30.5, 34.6)	226	35.1 (31.3, 39.1)	479	31.4 (29.0, 34.0)
High days of SARS‐CoV‐2 transmission ¶ , ,‖							.46
0 days	1238	62.2 (60.0, 64.4)	352	60.2 (56.0, 64.3)	886	63.1 (60.4, 65.7)
1‐7 days	464	22.1 (20.3, 24.1)	143	23.8 (20.4, 27.6)	321	12.4 (19.3, 23.7)
8‐14 days	310	15.6 (14.0, 17.4)	95	6.0 (13.1, 19.4)	215	15.5 (13.6, 17.6)

Table shows unweighted counts (N), weighted overall and column percentages (%), and weighted 95% confidence intervals (CI). Academic grade change represents change in letter grades since before the start of the COVID‐19 pandemic (before February 2020). p‐Values <.05 were considered statistically significant.

SY, school year.

Academic grade change represents change in letter grades since before the start of the COVID‐19 pandemic (before Feb. 2020). Grade change could not be computed for n = 24 survey respondents because they reported “none of these grades” or “not sure” for 1 or both questions about academic grades (overall sample size (2152) minus surveys without grade change information (24) = 2128).

Based on survey respondent's self‐reported state of residence.

Based on survey respondent's self‐reported location of school. This information was missing for n = 188 respondents.

2018 CDC/ATSDR Social Vulnerability Index (SVI) for the county where school is located. Low = counties in bottom one‐third, or tertile, of SVI scores (lowest vulnerability). Moderate = counties in middle tertile of SVI scores. High = counties in top tertile of SVI scores (highest vulnerability).

Could not be computed for n = 188 survey respondents due to missing school location information.

Number of days that the county where the school is located was designated as having a high SARS‐CoV‐2 transmission level (8‐14 days).

Table A2

Multivariable Logistic Regression Results for Having Same or Improved Academic Grades (Compared to Prepandemic) and Community Characteristics, Subpopulated by Student Learning Modality (In‐Person, Hybrid, or Remote/Virtual) in SY 2020‐2021

			Student Learning Modality
	Overall (N = 2012)		In‐person (n = 403)		Hybrid (n = 830)		Remote/virtual (n = 779)
	OR (95% CI)	p‐Value	OR (95% CI)	p‐Value	OR (95% CI)	p‐Value	OR (95% CI)	p‐Value
Community characteristics
Census region*
Midwest	Ref.		Ref.		Ref.		Ref.
Northeast	1.14 (0.81, 1.61)	.440	1.56 (0.52, 4.70)	.427	1.18 (0.74, 1.88)	.490	1.10 (0.59, 2.05)	.755
South	1.43 (1.06, 1.93)	.021	1.56 (0.72, 3.36)	.260	1.58 (1.01, 2.46)	0.045	1.42 (0.82, 2.47)	.215
West	1.30 (0.92, 1.85)	.141	1.35 (0.51, 3.59)	.550	1.48 (0.88, 2.50)	.144	1.23 (0.67, 2.31)	.486
Locality †
Rural	Ref.		Ref.		Ref.		Ref.
Suburban	0.96 (0.72, 1.28)	.793	0.60 (0.29, 1.24)	.167	0.91 (0.60, 1.36)	.632	1.38 (0.82, 2.31)	.226
Urban	0.95 (0.69, 1.31)	.760	0.56 (0.24, 1.28)	.168	1.02 (0.64, 1.64)	.929	1.23 (0.71, 2.11)	.461
Social vulnerability index ‡ , §
Low	Ref.		Ref.		Ref.		Ref.
Moderate	0.75 (0.57, 0.98)	.032	1.17 (0.60, 2.32)	.640	0.65 (0.44, 0.96)	.031	0.79 (0.51, 1.24)	.305
High	0.72 (0.54, 0.96)	.023	0.66 (0.32, 1.35)	.253	0.74 (0.48, 1.16)	.193	0.74 (0.47, 1.14)	.171
High days of SARS‐CoV‐2 transmission § , ‖
0 days	Ref.		Ref.		Ref.		Ref.
1‐7 days	0.89 (0.68, 1.17)	.400	0.57 (0.28, 1.18)	.130	0.72 (0.48, 1.09)	.113	1.23 (0.79, 1.92)	.358
8‐14 days	0.94 (0.70, 1.25)	.659	1.17 (0.51, 2.68)	.705	0.82 (0.54, 1.26)	.368	0.92 (0.55, 1.53)	.753

Overall model was not subpopulated by learning mode. Significant findings (p < .05) are bolded. Models also included student and school characteristics listed in Table 3. Referent group for the outcome variable, academic grade change, is declined.

CI, confidence interval; SY, school year.

Based on survey respondent's self‐reported state of residence.

Based on survey respondent's self‐reported location of school. This information was missing for n = 188 respondents.

2018 CDC/ATSDR Social Vulnerability Index (SVI) for the county where school is located. Low = counties in bottom one‐third, or tertile, of SVI scores (lowest vulnerability). Moderate = counties in middle tertile of SVI scores. High = counties in top tertile of SVI scores (highest vulnerability).

Could not be computed for n = 188 survey respondents due to missing school location information.

Number of days that the county where the school is located was designated as having a high SARS‐CoV‐2 transmission level (8‐14 days).

In‐Person Learning

Among in‐person students, race/ethnicity was the only measure significantly associated with grade change. When compared to non‐Hispanic white students, the odds of reporting same/improved grades compared to prepandemic were 65% lower among non‐Hispanic black students and 66% lower among non‐Hispanic other race students (p < .05).

Hybrid Learning

Among hybrid students, anxiety symptoms and substance use were the only measures significantly associated with grade change. The odds of reporting same/improved grades among students who reported anxiety symptoms were 47% less than students not reporting anxiety symptoms (p < .01). The odds of reporting same/improved grades among students who reported substance use were 40% less than students who did not report substance use (p < .05).

Remote/Virtual Learning

Having depression symptoms and receiving school‐provided educational services were the only measures significantly associated with grade change. The odds of reporting same/improved grades among students who reported depression symptoms were 62% less than the odds of students did not report depression symptoms (p < .001). Those who received educational services from school had 1.55 times the odds of reporting same/improved grades, compared to students not receiving these services (p < .05).

DISCUSSION

Approximately 20% of students attending school in‐person reported their grades declined since the start of the pandemic, with more students in the hybrid (about 30%) and remote/virtual (34%) groups reporting a decline. These findings are concerning given teachers might have been more lenient with grading during the 2020‐2021 school year, especially with virtual‐only students, , suggesting actual learning loss could be greater.

Public health experts have noted the potential for the pandemic to exacerbate existing racial/ethnic disparities in education and health. Learning modality may be a contributing factor. In a study of 12 US states, large reductions in test scores for the 2020‐2021 school year were observed compared to previous years, and this decline was significantly larger in districts with less access to in‐person schooling. US districts serving mostly black and Hispanic students and districts with high concentrations of limited English learners were less likely to provide primarily in‐person instruction and provided less instruction time in general than schools with lower proportions of minorities and English language learners. In this study, non‐Hispanic white students comprised 53% of the sample but only 39% of virtual/remote learners. The finding that full‐time virtual learning was more prevalent among black and Hispanic students is consistent with other studies. ,

While grade change was significantly associated with race/ethnicity in bivariate models, adjusted models showed this was only the case for in‐person students, with black and non‐Hispanic students of other races having lower odds of reporting same/improved grades (compared to prepandemic) than white students. One potential explanation that warrants further research is the role of teacher implicit bias to determine if implicit bias played a greater role for in‐person compared to virtual instruction during the pandemic, possibly influencing how much race/ethnicity minority students attending school in‐person felt connected to school. Research indicates that students who feel connected to school and perceive teachers and school rules/policies as fair, have higher academic achievement. These biases might be more likely to emerge when interacting face‐to‐face, and there is some evidence that counties where teachers hold higher levels of implicit and explicit racial bias have larger adjusted white/black test score inequalities.

Results consistently reflect that virtual learning (full‐ or part‐time) was challenging for students in all racial/ethnic groups but was disproportionately experienced by racial/ethnic minority students. Among remote/virtual learners, students receiving educational services from school had higher odds of maintaining or improving grades, indicating virtually delivered academic support did help students keep up during the pandemic. However, it is unclear whether students had equitable access to these supports which involves both school‐provision of services and individual capacity to receive (eg, highspeed internet, WiFi‐enabled device, private/dedicated space). Receipt of school‐provided, health‐related services (eg, occupational therapy, speech therapy) was not related to grade change. One possible explanation is that tutoring supports may have been supplemental in some districts and not legally required whereas occupational and speech therapy may have been a part of required Individual Education Plans. We are not aware of other studies addressing the provision of virtual school‐provided learning supports and academic outcomes, although successes and challenges with remote learning for K‐12 schools have been documented. , These results point to the importance of including strategies for continuity in academic support services in school emergency operations plans.

Roughly 1 in 5 students reported currently experiencing symptoms of depression or anxiety, corroborating results from the COVID Experiences Study (COVEX). , The learning experience of students with symptoms of mental health conditions varied by learning modality. Efforts to improve and sustain school connectedness, which has been shown to be protective for adolescent mental health, might help attenuate impacts of learning modality on mental health. Additionally, protective factors not included in our survey (eg, parental supervision, family support/connectedness) may have buffered potentially deleterious effects of disruptions to in‐person schooling on academic achievement.

IMPLICATIONS FOR SCHOOL HEALTH

The COVID‐19 pandemic has resulted in unprecedented disruptions to K‐12 education. This study provides evidence that many students may have been struggling to maintain their academic grades while also grappling with their mental health. As many as 20% of students reported symptoms of anxiety and/or depression and roughly 30% reported their grades had worsened compared to prepandemic, with higher proportions of decline among remote/virtual learners. Helping students to develop positive coping skills and address nonacademic barriers to learning may be important for successful return to in‐person learning. K‐12 schools may find that some student populations have fallen behind academically because of the pandemic, particularly among students who engaged in virtual learning, suggesting additional support and accommodations are needed.

These findings highlight the importance of keeping schools open for in‐person learning, when feasible. However, if schools need to pivot to virtual instruction in response to an emergency, it is critical that K‐12 school administrators have plans in place to lessen detrimental effects on student mental health and academic progress. Ensuring that schools are prepared to offer educational services and learning accommodations virtually might help students at least maintain their grades during such disruptions.

Schools may need to increase their capacity to provide educational supports to students struggling academically, including effective means to identify those with the greatest need. Likewise, additional resources to support and promote mental health and well‐being may be useful, especially for those students who have engaged in remote/virtual learning during the pandemic. The American Rescue Plan and Elementary and Secondary School Emergency Relief Fund presents an opportunity to more equitably strengthen the capacity of educational systems to recover from disruptions related to COVID‐19 and other emergencies. More equitable policies and practices in US schools related to educational and mental health support will further address students disproportionately impacted by the pandemic and continue to strengthen teaching and learning.

Limitations

The following limitations are noted. First, survey data were self‐reported and could reflect social desirability bias. Second, a convenience survey sample was used and may not be representative of all middle and high school students attending K‐12 schools in the US. Third, because the survey was related to COVID‐19 and there was a need to collect data rapidly, some survey items had not been previously evaluated in terms of reliability and validity (we note in the methods any previous validations or use of measures in other research). Fourth, the survey was conducted online so the experiences of students without reliable internet and computer access are not represented. Fifth, this was a cross‐sectional study design and conclusions about causality of associations cannot be drawn. Sixth, grades are subjective measures of performance and teachers could have adjusted their grading methods for students in different learning modalities (as documented elsewhere , ). Seventh, school district‐level SVI data were not available and social vulnerability characteristics may vary throughout a given county (and level of resources available to private schools could also vary). Additionally, since data were collected solely at the student level, results might not fully characterize the extent to which schools provided academic and health support services. Finally, absenteeism, COVID‐19 illness (for students, family members, caregivers), and COVID‐19 deaths (for family members, caregivers) were not collected but could have affected grades.

Conclusions

Overall, larger proportions of students who attended school in‐person full‐time during the 2020‐2021 school year maintained or improved their grades, compared to students attending remotely/virtually (part‐time or full‐time). This finding is concerning as only about 20% of students attended school in‐person full‐time in 2020‐2021. Since virtual learning has been shown to be more prevalent among lower‐income and racial/ethnic minority students, widening disparities in academic achievement following long stretches of remote instruction remains essential to address. ,

Models examined by learning modality provide tailored insights and have the potential to inform preparedness planning in anticipation of future shifts to virtual learning. School districts with plans to operate virtually in emergency situations in the future may need to be prepared to offer students additional academic support. Furthermore, school administrators, teachers, and other staff may need to be prepared to recognize if students who experience academic grade declines are also experiencing symptoms of depression or anxiety.

While a shift to virtual learning may have been protective against SARS‐CoV‐2 transmission among K‐12 student populations, it has been linked with lower school connectedness which might contribute to academic disengagement. Reduced access to in‐person learning has also been associated with poorer learning and negative mental health and behavioral outcomes in children. , While online teaching and learning has been shown to be effective with trained faculty and high‐quality instructional design (eg, online college/university courses), emergency remote teaching stemming from a crisis can diminish instruction quality.

Human Subjects Approval Statement

This activity was determined to meet the definition of research involving human subjects and Institutional Review Board approval was provided by the Advarra Institutional Review Board (See 45 C.F.R. part 46; 21 C.F.R. part 56).

Conflict of Interest

All authors of this article declare they have no conflicts of interest. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.