COVID-19, college academic performance, and the flexible grading policy: A longitudinal analysis.

I use an unbalanced panel of over 11,000 academic records spanning from Spring 2017 to Spring 2020 to identify the difference in effects of the COVID-19 pandemic across lower- and higher-income students' academic performance. Using difference-in-differences models and event study analyses with individual fixed effects, I find a differential effect by students' pre-COVID-19 academic performance. Lower-income students in the bottom quartile of the Fall 2019 cumulative GPA distribution outperformed their higher-income peers with a 9% higher Spring 2020 GPA. This differential is fully explained by students' use of the flexible grading policy with lower-income ones being 35% more likely to exercise the pass/fail option than their counterparts. While no such GPA advantage is observed among top-performing lower-income students, in the absence of the flexible grading policy these students would have seen their GPA decrease by 5% relative to their counterfactual pre-pandemic mean. I find suggestive evidence that this lower performance may be driven by lower-income top-performing students experiencing greater challenges with online learning. These students also reported a higher use of incompletes than their higher-income peers and being more concerned about maintaining (merit-based) financial aid.

Introduction

There is mounting evidence that the COVID-19 pandemic, with its subsequent closing of schools and campuses and move to online teaching, may be widening socio-economic educational gaps (Andrew et al., 2020, Aucejo et al., 2020, Engzell et al., 2021, Maldonado and De Witte, 2020, Rodríguez-Planas, 2022, Chetty Raj et al., 2020, Sass and Goldring, 2021). The digital divide and uneven access to online learning resources is one mechanism underlying the greater learning losses among lower-income students (Barnum and Bryan, 2020, Bacher-Hicks et al., 2021, Altindag et al., 2021). Other factors contributing to the learning delay include disadvantaged students’ lower access to physical learning space and conducive learning environment (Andrew et al., 2020), and higher stress and anxiety due to greater uncertainty and disruptions (Aucejo et al., 2020, Barnum and Bryan, 2020, Rodríguez-Planas, 2022, Jaeger et al., 2021).

At the same time, because online learning requires more discipline and self-regulated learning than traditional in-person learning, it would be reasonable to expect the educational gap between low- and high-performing students to widen if basic skills are necessary to acquire additional skills (Cunha and Heckman, 2007). Using pre-COVID-19 estimates of learning losses from extended absences from school (i.e., summer months and natural disasters), Kuhfeld et al. (2020) project substantial learning losses from the COVID-19 pandemic, especially for low-performing students. Using data from primary schools in Germany, Grenewig et al. (2020) find that, after COVID-19 school closures, higher-performing students spent more time on school-related activities daily than their lower-performing peers. Kofoed et al. (2021) find that West Point students randomized in an online version of a “Principles of Economics” course in Fall 2020 underperformed in assignments and exams relative to those in the in-person sessions. Using data from Virginia community college system, Bird et al. (2020) find a decrease in course completion in the courses students started in-person in Spring 2020 as compared to those they started online. Both studies find that the losses were greater among the low-performing students.

Despite this evidence, little is known on how the pandemic affected college students’ academic performance during the Spring 2020 and whether it had any differential effect on lower-income students’ academic performance relative to that of their higher-income peers. This is the main objective of this paper. This study provides novel evidence on the differential effects of the pandemic on Spring 2020 GPA and credits taken, earned, and failed by students’ pre-pandemic income among students enrolled in Queens College. To do so, it uses an unbalanced panel of over 11,000 academic records spanning from Spring 2017 to Spring 2020. The study further identifies the mechanisms driving its main findings by analyzing students’ Spring 2020 transcripts as well as their responses to a rich online survey.

Queens College (QC) is a four-year college in the City University of New York (CUNY) system. Considered one of the most affordable colleges in the country—with a median undergraduate tuition of $6,530—, QC is an urban college with a socially vulnerable and ethnically diverse student population located in the borough of Queens in New York City. The identification strategy relies on both difference-in-differences models and event study analyses. In addition, to control for time-invariant unobserved heterogeneity, I exploit within-student variation by controlling for student fixed effects.

During Spring 2020, higher-income students earned a 13.4% higher Spring 2020 GPA relative to their pre-pandemic mean. In contrast with students’ academic expectations for Spring 2020 (Aucejo et al., 2020, Rodríguez-Planas, 2022), lower-income students outperformed their wealthier peers as they earned a 5.1% higher GPA. However, average effects hide important differences by pre-COVID-19 academic performance.

Lower-income students in the bottom quartile of the 2019 Fall cumulative GPA distribution outperformed their higher-income counterparts with a 9% higher Spring 2020 GPA relative to the counterfactual’s pre-pandemic mean. This higher performance is strongly associated with the flexible grading policy as it vanishes when I use the GPA students would have earned in the absence of the flexible grading policy. Indeed, transcript data reveal that lower-income students from the bottom quartile were 35% more likely to exercise the pass/fail option than their higher-income counterparts. I further find suggestive evidence that greater concerns with maintaining financial aid among the bottom-performing lower-income students (relative to their peers) may have driven their differential prevalence of pass/fail grade.

While no such GPA advantage is observed among top-performing lower-income students, in the absence of the flexible grading policy these students would have underperformed relative to their higher-income counterparts as their GPA would have been 4% lower (relative to the pre-pandemic mean for the comparison group of 3.718). To put it differently, in the absence of flexible grading, lower-income top-performing student would have seen their GPA decrease by 5% relative to their counterfactual pre-pandemic mean. I find suggestive evidence that this lower performance may be driven by lower-income top-performing students experiencing greater challenges with online learning than their wealthier peers. These students also reported a higher use of incompletes and being more concerned about maintaining (merit-based) financial aid. Transcript data also reveal that they were 57% less likely to exercise the pass/fail option than bottom-performing lower-income students relative to the differential observed between their wealthier counterparts.

To the best of my knowledge, this paper is the first to use higher-education administrative records and transcript data to study the short-run effects of the pandemic on college students’ academic performance using a student fixed effects model. This work relates to at least the following two strands of literature. First, it is close to studies analyzing the effect of violent conflicts or natural disasters on students’ academic performance (Brück et al., 2019, Sacerdote, 2012) or of economic recessions on students’ earnings after graduation (Oreopoulos et al., 2012, Fernández-Kranz, 2018). While these studies focus on the effects of events long after they occurred, I study the immediate effects of the event.

Second, the current paper contributes to a nascent literature analyzing the consequences of the COVID-19 pandemic on college education. In contrast with Aucejo et al. (2020) and Rodríguez-Planas (forthcoming), which focus on students’ self-perceived challenges, I find that the immediate effects of the COVID-19 pandemic on academic performance are not only positive but seem to benefit most lower-performing lower-income students. While Kofoed et al. (2021) study the impact of switching from in-person to online learning on grades during the pandemic, their analysis is confined to an Economics course during Fall 2020 with no temporal analysis. Bird et al. (2020) and Altindag et al. (2021) also estimate the effect of the pandemic on students’ academic performance due to the unexpected switch to online learning during Spring 2020. However, both papers focus on universities with well-established online education programs, and their identification strategy relies on comparing academic performance of students who took courses in both types of instruction modes before and after the pandemic. Because online teaching was practically non-existent prior to the pandemic at QC, estimates in the current study pick up both the switch to online teaching and other disruptions caused by the pandemic. By merging administrative records with transcript data and students’ responses to a survey on COVID-19 challenges, I document differential effects of the pandemic given students’ pre-pandemic income and pre-pandemic academic performance more broadly. Importantly, my analysis reveals students’ differential use of the flexible grading policy based on their financial and academic needs.

Data

The data in this paper come from three different sources. I merged individual administrative academic records from QC with survey data on students’ challenges collected during the early months of the pandemic. To better understand students’ use of the flexible grading policy, I also used transcripts level data.

Most of the analysis focuses on an unbalanced panel of 11,443 academic records from 2,817 students spanning from Spring 2017 or later (if the student enrolled in QC at a later date) to Spring 2020.2 For each semester, I observe students’ semester GPA; credits taken, which do not include courses officially withdrawn as they do not affect the GPA; credits earned; and credits for which a failing grade was earned. In regular years, the latter include credits from: (1) courses unofficially withdrawn (where the student stopped attending and never withdrew); (2) courses with an F grade; and (3) courses with a grade of Failed Incomplete, which is the grade assigned when an incomplete is not resolved by the following semester. In Spring 2020, this category also includes credits for courses with a grade of No Credit (NC). Other information available in QC administrative records includes students’ sex, age, race, and ethnicity. In addition, I also observe the following information collected at the beginning of Spring 2020: students’ major, class level (indicating whether the student is a freshman, sophomore, junior, senior, or in graduate school), Fall 2019 cumulative GPA, part-time student status, and whether the student had ever received the federal Pell grant. I use information on ever Pell grant receipt to define lower-income students because most Pell grant money goes to students with a total family income below $20,000.

In addition, Spring 2020 transcript data provide information on whether the student exercised the pass/fail option (no-letter grade), took an incomplete, or simply removed the class record after the end of the semester (the NC grade). I use such information to estimate both the prevalence and intensity of such options during the Spring 2020. Transcript data also inform us on whether the student withdrew a course officially or unofficially. If the student officially withdrew the course, she did so within the first nine weeks of the semester, and her GPA was not affected. If the student never withdrew the course but stopped attending, it is considered as an unofficially withdraw, and the student received a failing grade. I use such information to estimate both the prevalence and intensity of courses withdrawn officially and unofficially. Importantly, since transcript data also include the initial grade received before changes were made, and the credits each course was worth, I use such information to calculate the students’ Spring 2020 GPA in the absence of the flexible grading policy.

To further identify potential mechanisms behind these findings, I use the students’ responses to an online survey on their experiences during Spring 2020. The survey was sent to all students enrolled in Spring 2020, and it was fielded between Friday, July 24th and Friday, September 18th 2020. The response rate of 23% is higher than the usual response rate on CUNY online surveys of 13%, and the response rate of 10% to 12% obtained around the same time in 28 universities (Jaeger et al., 2021). The sample in the current paper is almost twice as large as that of other post-COVID-19 higher-education survey studies (Aucejo et al., 2020).

As seen in Table 1 , I observe small differences between students in my sample (columns 1 and 2) and the overall QC student population (column 4) in the racial/ethnic distribution, the share of part-time students, the distribution of majors3 , and the share of Pell recipients. For example, the share of students who ever received the Pell grant in my sample is 57% if only undergraduate students are considered, which is not far from the 55% observed at the college level.

Table 1

Descriptive statistics.

		Queens College
	Higher-income students	Lower-income students	Difference (2) minus (1)		Registered in Fall 2019a
	(1)	(2)	(3)		(4)
Baseline characteristics
Female	0.6869	0.6666	−0.020	(0.018)	0.568
Black	0.1166	0.1163	−0.000	(0.012)	0.086
Asian	0.2485	0.4063	0.158***	(0.017)	0.285
Hispanic	0.2543	0.3457	0.091***	(0.017)	0.284
White	0.3812	0.1446	−0.237***	(0.016)	0.269
18 years old	0.1376	0.1150	−0.023	(0.012)	0.163
19 years old	0.0913	0.1356	0.044***	(0.012)	0.098
20 to 22 years old	0.2659	0.3099	0.019	(0.012)	0.312
23 to 24 years old	0.1208	0.1296	0.004	(0.012)	0.136
25 to 29 years old	0.1579	0.1666	0.009	(0.014)	0.158
30 to 44 years old	0.1615	0.1260	−0.036**	(0.013)	0.105
Over 45 years old	0.0652	0.0227	−0.042***	(0.008)	0.028
US born	0.4884	0.4001	−0.088***	(0.019)	0.677
Ever Pell receipt	0	1			0.547b
ESL	0.1754	0.2608	0.086***	(0.015)	0.357b
First-generation	0.2848	0.4305	0.146***	(0.018)	--
Transfer student	0.1652	0.2683	0.103***	(0.015)	0.555
Employed	0.7268	0.6838	−0.043*	(0.017)	--
Part-time student	0.4608	0.2431	−0.218***	(0.017)	0.351
Freshman	0.0341	0.0048	−0.029***	(0.005)	--
Sophomore	0.2775	0.4174	0.140***	(0.018)	--
Junior	0.1413	0.1405	−0.001	(0.013)	--
Senior	0.2058	0.2803	0.075***	(0.016)	--
Graduate	0.2775	0.0798	−0.198***	(0.014)	--
Online enrollment	--	--	--	--	0.023
Fall 2019 GPA	3.215	2.973	−0.242***	(0.028)	--
Sample size	1,380	1,452	2,832		19,923

Note: Lower-income students are those who ever received the Pell grant based on QC administrative records.

Standard errors are reported in parentheses in column 3. Column 3 presents the coefficient on the low-income dummy from a regression model with no other controls.

Significant at the: ***1 percent level, ** 5 percent level, *10 percent level.

Source: .

Excludes graduate students.

There is a higher share of females (68% versus 57%) and older-than-25-years-old students (35% versus 29%) than the overall QC population, and a lower share of US born (44% versus 68%), English second-language learners (ESL) learners (22% versus 36%), and transfer students (22% versus 55%). The lower rate of transfer students reflects the lower engagement of these students to regular college life as they frequently combine college with either part-time or full-time employment.

CUNY is known to be an institution that educates some of the poorest students in the country. It is also known to have a very diverse student population. Hence, it is not surprising that QC students are more racially diverse than students from the largest public university in each state. For example, only 27% of QC students are non-minority students compared to an average of 61% of non-minority students in the largest public universities in each state (Aucejo et al., 2020).

Comparing columns 1 and 2 reveals that lower-income students (defined as those who ever received a Pell grant) are more likely to be Asians or Hispanics than higher-income students. They are also more likely to be first-generation college students, transfer students, and ESL students, and less likely to be US born than higher-income students. They also have a lower Fall 2019 cumulative GPA. Given the Pell-grant requirements, lower-income students are younger, and less likely to be graduate students or study part-time than higher-income students.

Statistical methods

To estimate the differential effect of the COVID-19 pandemic on lower-income students’ academic performance, I estimate the following difference-in-differences model with individual fixed effects:where Y is the outcome of interest (for example, semester GPA) for student i in semester s and year t. Spring2020 is a dummy equal 1 if the academic record is for Spring 2020 and 0 prior to that. Low-Income is equal to 1 if student i ever received the Pell grant and 0 if the student never received the Pell grant. φ represents the individual fixed effects, Fall is a dummy equal 1 if the academic record is for the Fall semester and 0 if it is for the Spring semester, and Year represents the year fixed effects. Standard errors are clustered at the student level.

The coefficient of interest, β, captures the differential post-pandemic effect on the outcome, Y, for lower-income students relative to their higher-income peers. Note that the individual fixed effects, φ, absorb the lower-income indicator (as well as all the other time-invariant observable and unobservable characteristics). The coefficient β captures how the academic performance of higher-income students changed in Spring 2020 when the COVID-19 pandemic hit. As identification comes from comparing outcomes from the same student before and after the pandemic, there is no need to control for time-invariant observable characteristics. The Fall semester dummy controls for semester-specific characteristics, and the year fixed effects control for year differences over time.

The critical identifying assumption is that there are parallel trends in the outcome variable across both groups (lower- versus higher-income students). To assess the validity of this assumption, I check for pre-existing diverging trends using the following event-study framework:where S is a dummy which takes value 1 if the outcome is observed in jth semester before (-j) or after (+j) January 27th 2020, which is when Spring 2020 began, and 0 otherwise. Fall 2019 dummy is the omitted semester. In the absence of any pre-existing differential pre-trends between lower- and higher-income students, the estimated coefficients corresponding to the semesters prior to the Spring 2020 would not be statistically different from zero.

Because average effects may hide differences by students’ pre-COVID-19 academic performance, I also estimate the differential effect for students in the different quartiles of the 2019 cumulative GPA distribution using the following difference-in-differences model with individual fixed effects: where is a dummy variable which takes the value 1 if the Fall 2019 cumulative GPA of student i is in the q quartile—the reference category is the first quartile. All the other covariates have been previously defined. In this specification, the individual fixed effects, φ, absorb the lower-income indicator, the quartile indicators, , and the interaction of the low-income indicator with the quartile indicators, , as well as all the other time-invariant observable and unobservable characteristics.

The coefficient captures the change in outcome during Spring 2020 experienced by higher-income students in the bottom quartile. The sum of and , , captures the change in outcome during Spring 2020 experienced by higher-income students in the q quartile. And the coefficient measures the size of the differential effect in the change in outcome before and after January 27th 2020 for higher-income students in q quartile relative to similar students in the bottom quartile.

To identify changes by income level, the coefficient captures the differential effect in the change in outcome after the pandemic between lower- and higher-income students in the bottom quartile. The sum of and , (, captures the differential effect in the change in outcome after the pandemic between lower- and higher-income students in q quartile. The coefficient measures the differential effect in the change in outcome post-pandemic between lower- and higher-income students in q quartile relative to that experienced between lower- and higher-income students in the bottom quartile. At the bottom of Panel B of Table 2 , I display and ( for the top three quartiles (q = 2, 3, and 4). Because Eq. (3) is an individual fixed effects model, time-invariant individual unobserved heterogeneity is held constant.

Table 2

Academic performance during Spring 2020 by low-income status and pre-pandemic academic performance, individual fixed-effects estimates, outcomes from academic records.

	Semester GPA after exercising Flex. Grading Option	Credits taken after removing officially withdrawn courses	Credits earned officially	Credits for which a failing grade was earned	Semester GPA before exercising Flexible Grading Option
Panel A: Average effect	(1)	(2)	(3)	(4)	(5)
Spring 2020 (β_1)	0.405*** (0.030)	0.392*** (0.115)	0.472*** (0.125)	−0.081 (0.060)	0.228*** (0.030)
Spring 2020 * Low Income (β_2)	0.155*** (0.036)	−0.106 (0.144)	0.087 (0.159)	−0.192** (0.077)	0.025 (0.036)
Pre-mean higher-income students	3.014 (1.116)	10.204 (5.076)	9.530 (5.154)	0.674 (1.860)	3.014 (1.116)
Observations	11,443	11,443	11,443	11,443	11,443
Students	2,817	2,817	2,817	2,817	2,817
R2	0.077	0.005	0.008	0.003	0.0246
Panel B: By Fall 2019 cumulative GPA quartiles
Spring 2020 (β_1)	0.907*** (0.072)	0.822*** (0.240)	1.528*** (0.287)	−0.706*** (0.179)	0.554*** (0.075)
Spring 2020*Q2 (γ_2)	−0.382*** (0.090)	−0.246 (0.331)	−0.811** (0.371)	0.564*** (0.204)	−0.736*** (0.094)
Spring 2020 *Q3 (γ_3)	−0.609*** (0.084)	−0.512* (0.307)	−1.384*** (0.350)	0.873*** (0.195)	−0.466*** (0.087)
Spring 2020 *Q4 (γ_4)	−0.810*** (0.078)	−0.762*** (0.295)	−1.615*** (0.336)	0.853*** (0.185)	−0.600*** (0.082)
Spring 2020 *Low Income (β_2)	0.166* (0.087)	−0.238 (0.4304)	−0.033 (0.360)	−0.204 (0.228)	−0.024 (0.091)
Spring 2020 * Low Income *Q2 (δ_2)	−0.149 (0.111)	−0.238 (0.427)	−0.362 (0.480)	0.124 (0.266)	0.014 (0.117)
Spring 2020 * Low Income *Q3 (δ_3)	−0.161 (0.105)	0.397 (0.409)	0.215 (0.462)	0.182 (0.257)	−0.039 (0.111)
Spring 2020 * Low Income *Q4 (δ_4)	−0.222** (0.103)	−0.036 (0.418)	−0.277 (0.470)	0.241 (0.248)	−0.120 (0.108)
Constant	2.936*** (0.014)	11.031*** (0.053)	10.191*** (0.055)	0.840*** (0.032)	2.934*** (0.014)
(β_1+γ_2)	0.525*** (0.058)	0.576** (0.239)	0.717*** (0.249)	−0.141 (0.108)	0.178*** (0.057)
(β_1+γ_3)	0.298*** (0.048)	0.311 (0.201)	0.144 (0.213)	0.167* (0.090)	0.088* (0.047)
(β_1+γ_4)	0.097*** (0.036)	0.060 (0.184)	−0.087 (0.192)	0.147** (0.067)	−0.046* (0.047)
(β_2+δ_2)	0.017 (0.070)	−0.475 (0.300)	−0.395 (0.317)	−0.080 (0.136)	−0.009 (0.073)
(β_2+δ_3)	0.005 (0.060)	0.159 (0.274)	0.182 (0.290)	−0.023 (0.119)	−0.063 (0.061)
(β_2+δ_4)	−0.056 (0.055)	−0.274 (0.287)	−0.310 (0.302)	0.036 (0.098)	−0.144** (0.057)
Observations	11,443	11,443	11,443	11,443	11,443
Students	2,817	2,817	2,817	2,817	2,817
R2	0.111	0.007	0.015	0.014	0.042

Notes: Spring 2020 is a dummy equal 1 if the outcome is measured during Spring semester 2020, and 0 otherwise. Low Income is a dummy equal 1 if the student ever received the Pell grant, and 0 otherwise. Q1 is a bottom quartile dummy equal to 1 if the student’s cumulative GPA is below 2.64 and equal 0 otherwise. Q2 is a second quartile dummy equal to 1 if the student’s cumulative GPA ranges between 2.64 and 3.23, both included, and equal 0 otherwise. Q3 is a third quartile dummy equal to 1 if the student’s cumulative GPA ranges between 3.24 and 3.71, both included, and equal 0 otherwise. Q4 is a third quartile dummy equal to 1 if the student’s cumulative GPA is higher than 3.71, and equal 0 otherwise. Panel A reports individual fixed-effects estimates associated with post-pandemic and low- versus high-income students on the dependent variables indicated in column headings. I estimate Eq. (1) described in the main text. Panel B reports individual fixed-effects estimates associated with post-pandemic and low- versus high-income students from different cumulative Fall 2019 GPA quartiles on the dependent variables indicated in column headings. I estimate Eq. (3) described in the main text. Standard errors clustered at the individual level are reported in parentheses. All regressions include a Fall-semester indicator and year 2017 and year 2018 dummies.

*, *,**,*** Estimate significantly different from zero at the 0.1 level, 0.05 level, or 0.01 level.

Finally, when analyzing pre-pandemic income and academic performance heterogeneity effects with transcript or survey data, which is only available for Spring 2020, I estimate the following equation using ordinary least squares (OLS):where is the student’s i outcome during Spring 2020. is a vector of for all the baseline characteristics listed in Table 1 and students’ major. Importantly, I control for students’ pre-pandemic academic performance with the Fall 2019 semester GPA. The coefficient α captures the association between being a lower-income student in the bottom quartile and the students’ outcome relative to their higher-income counterfactual. Similarly, (α  captures the association between being a lower-income student in the q quartile and the students’ outcome relative to their higher-income peers. The coefficient measures the differential effect in the in Spring 2020 outcome between lower- and higher-income students in q quartile relative to that experienced between lower- and higher-income students in the bottom quartile.

Average effects by pre-pandemic income status

Higher-income students’ academic performance improved in Spring 2020 as students took and earned more courses than before the pandemic. They also got higher grades. As seen in Panel A of Table 2, the effect of the pandemic on higher-income students’ academic performance shows a 13.4% increase in semester GPA, a 4% increase in credits taken, and a 5% increase in credits earned during Spring 2020 relative to pre-pandemic means. All three effects are statistically significant at the 1% level.

It is plausible that the pandemic affected the number of credits taken because these do not include any courses officially withdrawn within the first nine weeks of the semester (that is, by March 30th 2020). While the flexible grading policy was effective on April 1st 2020, discussions on its content occurred during most of March and both faculty and students were aware of the policy’s consequences for students’ GPA. Hence, the Spring 2020 increase in credits taken is likely due to students not officially withdrawing courses by March 30th because they already knew that they would be able to exercise the NC grade without penalizing their GPA. In contrast, the flexible grading policy could not affect credits earned as the student is not awarded credit with the NC grade. Instead, the increase in credits earned can only be explained by an increase in passing versus failing grades during the Spring 2020.

The increase in GPA is also likely related to the fact that as many as 17.2% of higher-income students exercised the pass/fail option, 6.3% exercised the NC grade, and 5% exercised the incomplete option. Comparing the official GPA increase before and after the pandemic with the GPA increase students would have experienced in the absence of the flexible grading policy (columns 1 and 5, Panel A in Table 2), we observe that the GPA increase ended up being 5.9% higher relative to the average pre-pandemic GPA of 3.014 because of the policy.4 To put it differently, the flexible grading policy is responsible for 43.7% of the Spring 2020 GPA increase, . At the same time, it is important to underscore that, even in the absence of the flexible grading policy, the Spring 2020 GPA increased by 7.6% among higher-income students relative to the pre-pandemic mean suggesting that other factors beyond flexible grading may be at play here.

Moving to the differential post-pandemic effect on academic performance across lower- and higher-income students, lower-income students outperformed higher-income ones during Spring 2020 as they earned a 5.1% higher GPA and they failed 28% fewer credits (or exercised fewer NC grades) than their wealthier counterparts. Both effects are statistically significant at the 5% level or lower. Lower-income students’ higher relative academic performance during the first semester of the pandemic is largely associated with the flexible grading policy as the GPA differential by income status vanishes (β is close to zero and not statistically significant) when I estimate the effect of the pandemic with the GPA prior to students exercising the flexible grading option.

As seen in Panels A and D of Fig. 1 , after five semesters of pre-pandemic parallel trends, we observe an increase in lower-income students’ official GPA and a decrease in credits failed in the Spring 2020 relative to their counterparts. While in Panel B, lower-income students attempted less credits in Spring 2020 than their wealthier counterparts, they also did so in earlier semesters. In Panel C, the relative increase in credits earned between lower- and higher-income students is small and not precisely estimated. As discussed earlier, in the absence of the flexible grading policy, there is no differential effect in the semester GPA post-pandemic between lower- and higher-income students (as seen in Panel E).

Fig. 1

Semester GPA and Credits Differential by Income Level. Notes: These figures plot the coefficients on the interaction between the semester dummies and the low-income dummy (and the 95% confidence intervals) from an individual fixed-effects model as in Eq. (2) in the main text. The Fall 2019 semester dummy is the omitted semester. Standard errors are clustered at the individual level. In Panel A, the outcome is the official semester GPA. In Panel B and C, the outcome is credits taken and credits earned, respectively. In Panel D, the outcome is credits failed or NC earned. In Panel E, the outcome is the official semester GPA except for the Spring 2020, in which case I have replaced the official GPA with the GPA the student would have earned had the flexible grading policy not been available. I estimated this GPA using the courses the student enrolled in, and the grades originally earned in those courses (including official and unofficial withdraws) from transcript data.

Differential effects by pre-pandemic income status and academic performance

As seen in Panel B of Table 2, the higher relative average performance among lower-income students is solely driven by students in the bottom quartile of the distribution. These students’ Spring 2020 GPA increased an additional 9% relative to their higher-income counterparts’ pre-pandemic average. This differential effect is marginally statistically significant at the 10% level. Adding the overall post-pandemic effect, ( from Eq. (3) results in lower-income students from the bottom quartile of the distribution earning 1.073 additional points on their semester GPA in Spring 2020, an increase of 55% relative to their counterparts’ pre-pandemic GPA mean of 1.939. The coefficient ( is statistically significant at the 1% level (standard error = 0.051).

This higher relative GPA among low-performing students in the bottom quartile is strongly associated with the flexible grading policy as the differential vanishes when I use the GPA students would have earned in the absence of the flexible grading policy. Column 5, Panel B in Table 2 shows that β is close to zero, negative, and not statistically significant for this group of students. Indeed, as seen in column 3 of Table 3 , lower-income students from the bottom quartile were 8.1 percentage points more likely to exercise the pass/fail option than their higher-income counterparts. This effect represents a 35% increase in the prevalence of the pass/fail option relative to their counterparts’ prevalence of 22.9% and is statistically significant at the 5% level.

Table 3

Prevalence and intensity of flexible grading options by low-income status and pre-pandemic academic performance, OLS estimates, outcomes from transcript data.

	Exercise the “no credit” option	Number of “no credits” taken	Exercise the “pass/fail” (no letter grade) option	Number of “pass/fail” taken	Exercise the “incomplete” option	Number of “incompletes” taken	Officially withdraw a course	Number of credits officially withdrawn	Unofficially withdraw a course	Number of credits unofficially withdrawn (failed a grade)
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)
Constant	0.266*** (0.061)	0.399*** (0.107)	0.158** (0.070)	0.193* (0.112)	0.070 (0.043)	0.069 (0.055)	0.245*** (0.064)	0.348*** (0.106)	0.039 (0.028)	0.094 (0.062)
Quartile 2: 2.64 < GPA=<3.23	−0.129*** (0.037)	−0.166*** (0.061)	0.064 (0.044)	0.095 (0.081)	−0.057** (0.027)	−0.073* (0.037)	0.054 (0.036)	0.071 (0.049)	−0.027 (0.019)	−0.034 (0.026)
Quartile 3: 3.23 < GPA=<3.71	−0.161*** (0.044)	−0.226*** (0.073)	−0.021 (0.051)	−0.097 (0.086)	−0.074** (0.032)	−0.085* (0.048)	0.062 (0.042)	0.103 (0.063)	−0.047** (0.021)	−0.050 (0.034)
Quartile 4: 3.71 < GPA	−0.167*** (0.052)	−0.228*** (0.086)	−0.085 (0.058)	−0.210** (0.096)	−0.089** (0.037)	−0.113** (0.054)	0.043 (0.048)	0.085 (0.073)	−0.052** (0.024)	−0.051 (0.042)
Low income	−0.017 (0.031)	−0.002 (0.051)	0.081** (0.033)	0.081 (0.056)	−0.019 (0.022)	−0.018 (0.029)	0.030 (0.028)	0.039 (0.040)	−0.014 (0.016)	−0.014 (0.025)
Low income × Quartile 2	0.041 (0.036)	0.025 (0.059)	−0.136*** (0.046)	−0.211*** (0.031)	0.028 (0.028)	0.038 (0.036)	−0.52 (0.037)	−0.060 (0.051)	0.000 (0.019)	0.020 (0.027)
Low income × Quartile 3	0.026 (0.034)	0.018 (0.053)	−0.060 (0.044)	−0.037 (0.075)	0.030 (0.027)	0.027 (0.036)	−0.074** (0.034)	−0.093* (0.049)	0.019 (0.017)	0.021 (0.026)
Low income × Quartile 4	0.020 (0.032)	−0.003 (0.052)	−0.098** (0.041)	−0.104 (0.069)	0.045 (0.028)	0.053 (0.037)	−0.023 (0.033)	−0.041 (0.045)	0.019 (0.017)	0.020 (0.026)
(Low Inc) + (Low Inc *Q2)	0.024 (0.020)	0.028 (0.032)	−0.055* (0.033)	−0130** (0.065)	0.009 (0.017)	0.020 (0.022)	−0.023 (0.024)	−0.210 (0.032)	−0.014 (0.010)	−0.012 (0.011)
(Low Inc) + (Low Inc *Q3)	0.009 (0.014)	0.020 (0.018)	0.021 (0.030)	0.043 (0.051)	0.010 (0.015)	0.009 (0.022)	−0.045** (0.020)	−0.054* (0.029)	0.004 (0.007)	0.007 (0.007)
(Low Inc) + (Low Inc *Q4)	0.003 (0.010)	−0.000 (0.012)	−0.017 (0.025)	−0.024 (0.042)	0.025 (0.017)	0.035 (0.24)	0.007 (0.019)	−0.002 (0.023)	0.004 (0.005)	0.006 (0.006)
Number of students	2,831	2,831	2,831	2,831	2,831	2,831	2,831	2,831	2,831	2,831
R2	0.134	0.113	0.115	0.098	0.025	0.021	0.047	0.046	0.049	0.063

Notes: The table reports individual OLS on the low-income dummy, quartile dummies, and interactions between the two from a regression where the outcome is indicated in column headings. I estimate Eq. (4) in the main text. The bottom quartile is the omitted quartile. All regressions include a female, race and ethnicity indicators, a USA born indicator, a first-generation student indicator, a transfer-student indicator, an ESL-learner indicator, class-level indicators (including a graduate student indicator), major dummies, and Fall 2019 cumulative GPA. Robust standard errors are reported in parentheses.

*, **,*** Estimate significantly different from zero at the 0.1 or 0.05 level or 0.01 level.

A distinct pattern is observed between lower-income students in the top and bottom quartile of the distribution relative to the GPA gap by pre-pandemic performance of their higher-income counterparts as shown by a negative and statistically significant in column 1, panel B in Table 2. While the Spring 2020 GPA of top-performing higher-income students, increased by 0.097 points relative to their pre-pandemic mean of 3.718 (a statistically significant 2.6% increase), the Spring 2020 GPA of top-performing lower-income students only increased by half the size as (, and I could not reject the null hypothesis, . 5

Column 5 further reveals that, in the absence of the flexible grading policy, the GPA of lower-income top-performing students, ( would have been 4% lower than that of their higher-income counterparts relative to the pre-pandemic mean for the comparison group.6 This effect is statistically significant at the 5% level. To put it differently, in the absence of flexible grading, lower-income top-performing students’ Spring 2020 GPA would have been 0.189 points lower than their pre-pandemic average (, representing a 5% decrease relative to their counterfactual pre-pandemic mean GPA of 3.718. This effect is statistically significant at the 1% level.

Consistent with these estimates, the event study (shown in Fig. 2 ) paints a quite different picture for lower-income students in the bottom and top quartiles of the Fall 2019 cumulative GPA distribution relative to their wealthier peers. Panel A, which focuses on the bottom quartile, shows a lack of pre-pandemic trends followed by an increase in lower-income students’ Spring 2020 official GPA relative to their wealthier peers (LHS figure). However, no such increase is observed had the flexible grading policy not been available (RHS figure). In contrast, Panel B, which focuses on the top quartile, shows that the lack of pre-pandemic trends is followed by a decrease in lower-income students’ Spring 2020 GPA prior to students exercising the flexible grading policy relative to their wealthier peers (RHS figure), with no post-pandemic change observed with the official GPA (LHS figure). Appendix Fig. A.1 shows the event studies for the other outcomes and quartiles.

Fig. 2

Semester GPA by Income Level for Bottom- and Top-Performing Students. Before and After Exercising the Flexible Grading Policy. Notes: These figures plot the coefficients on the interaction between the semester dummies and the low-income dummy (and the 95% confidence intervals) from an individual fixed-effects model as in Eq. (2) in the main text using students in different quartiles of the Fall 2019 cumulative GPA distribution. Standard errors are clustered at the individual level. On the left-hand side, the outcome is the official semester GPA. On the right-hand side, the outcome is the official semester GPA except for the Spring 2020, in which case I have replaced the official GPA with the GPA the student would have earned had the flexible grading policy not been available. I estimated this GPA using the courses the student enrolled in, and the grades originally earned in those courses (including official and unofficial withdraws) from transcript data. Panel A shows estimates using students from the bottom quartile of the Fall 2019 semester cumulative GPA distribution. Panel B shows estimates using students from the top quartile. The Fall 2019 semester dummy is the omitted semester. Appendix Fig. A.1 shows estimates for two middle quartiles and for other outcomes, namely, credits taken, credits earned, and credits failed (or NC earned).

To address concerns that this differential pattern in the official Spring 2020 GPA may be driven by students differentially selecting majors according to their income, I re-estimate Eq. (3) adding the interaction of the Spring 2020 dummy with major dummies. Similarly, to determine whether this differential effect by income is partially driven by different income shares across years-in-college (freshmen, sophomore, junior, or seniors) or graduate school, I re-estimate Eq. (3) adding the interaction of the Spring 2020 dummy with dummies indicating years-in-college and enrollment in graduate school. Finally, because Spring 2017 academic outcomes are somewhat anomalous, I estimate Eq. (3) dropping observations from Spring 2017 semester. The findings in this study (shown in Appendix Table A.3) are robust to the above specifications suggesting that these alternative explanations are not driving this differential pattern by income.

It is worthwhile highlighting that, as for bottom-performing students, the flexible grading policy helped lower-income top-performing students increase their Spring 2020 GPA. In this case, the increase relative to their wealthier counterparts was of 0.088 points (from −0.144 to −0.056), the equivalent of 2.4% increase relative to the pre-pandemic counterfactual mean of 3.718. But, in contrast with lower-income students in the bottom quartile, the GPA for those in the top quartile did not increase relative to their wealthier peers once the flexible grading policy has been taken into account. Table 3 suggests that this may be due to the differential exercise of pass/fail option between top- and bottom-performing lower-income students relative to the differential prevalence among their wealthier peers. Indeed, column 3 shows that top-performing lower-income students were 9.8 percentage points less likely to exercise the pass/fail option than bottom-performing lower-income students relative to the differential observed between their wealthier counterparts. As 17.25% of higher-income students exercised the pass/fail option, this represents a reduction of 57%. This effect is statistically significant at the 5% level.

Yet, this does not explain the differential income effect of the pandemic on top-performing students before students exercised the flexible grading policy. Below, I explore plausible mechanisms behind lower-income top-performing students underperformance relative to their wealthier counterparts.

Mechanisms

Because of the disruptions that COVID-19 represented to higher education and the abrupt move to online learning, we would have expected academic performance of college students to drop. As explained in Section IV, this did not happen. Instead, the official Spring GPA of higher-income students increased by an average of 13.4% relative to the pre-pandemic and after controlling for time-invariant heterogeneity. While about two fifths of this increase is explained by the flexible grading policy, a 7.6% remains unexplained.

Several reasons could explain this improvement beyond students’ greater flexibility in their grading choices due to the flexible grading policy: (1) a different assessment process with easier exams and/or more lenient grading; (2) more difficult supervising process as exams were online, leading to potentially greater cheating; (3) an improvement in students’ learning strategies with online learning; (4) lower opportunity costs of studying due to less employment available; and (5) lower financial stress due to greater availability of emergency relief funds from the college or the government.

While I am unable to derive strict tests for the relative importance of these mechanisms, systematic heterogeneity of the effects of the pandemic across different groups of students based on pre-pandemic income and performance inequalities ought to provide suggestive evidence consistent with one mechanism but not with another.

It is plausible that changes in both faculty’s leniency, as well as exams’ assessment and supervision may be behind some of the higher post-pandemic GPA observed for the whole sample. However, given the findings on the role of the flexible grading policy, it is unlikely that they drive the observed academic-performance differences between lower- and higher-income students.

Assuming dynamic complementarity and self-productivity, we would expect multiplier effects of skills and ability (Cunha and Heckman, 2007). In such case, the online-learning costs would be lower for higher- than lower-performing students, consistent with recent findings (Grewenig et al., 2020, Kuhfeld et al., 2020, Kofoed et al., 2021, Bird et al., 2020). Yet, estimates from panel B of Table 2 indicate that the Spring 2020 GPA increase was inversely related to pre-pandemic performance.

To explore the other possibilities, I estimate model (4) using as outcomes students’ survey responses regarding: (1) their perception of having challenges with online learning; (2) their opportunity costs of studying, measured by whether they worked during the Spring 2020, and by whether they worked less due to COVID-19; (3) their receipt of emergency relief funds; and (4) their perception of how flexible grading influenced their grading choices.7 While these estimates are not capturing a causal relationship, they may present suggestive evidence of which mechanisms may be associated with the differential outcomes by income.

As seen in column 1 in Table 4 , top-performing lower-income students were 6.8 percentage points more likely to experience challenges with online learning than their higher-income counterparts. This effect is statistically significant at the 5% level. A similar income disadvantage is observed among students in third quartile, albeit the effect is only marginally significant. No such income differential effect is observed among students in the bottom half of the distribution.

Table 4

Students’ challenges and self-reported use of flexible grading by low-income status and pre-pandemic academic performance, OLS estimates, outcomes from survey data.

	Outcomes from survey data
	Online challenges during Spring 2020	Employed during Spring 2020	Worked less during Spring 2020	Received emergency relief assistance	Prevented from dropping a class	Chose pass/fail or no credit	Asked for an incomplete	Not aware of flexible grading policy	Challenge in maintaining financial assistance
	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)
Constant	0.697*** (0.072)	0.381*** (0.079)	0.362*** (0.079)	0.271*** (0.076)	0.343*** (0.080)	0.516*** (0.084)	0.142*** (0.046)	0.126** (0.054)	0.452*** (0.094)
Quartile 2: 2.64 < GPA=<3.23	0.053 (0.043)	−0.027 (0.045)	−0.031 (0.042)	0.032 (0.047)	−0.016 (0.046)	0.087* (0.048)	−0.060** (0.028)	−0.049 (0.026)	0.128** (0.052)
Quartile 3: 3.23 < GPA=<3.71	0.033 (0.053)	−0.106** (0.054)	0.003 (0.052)	0.083 (0.056)	−0.006 (0.056)	0.058 (0.058)	−0.099*** (0.032)	−0.057 (0.043)	0.152** (0.063)
Quartile 4: 3.71 < GPA	0.060 (0.061)	−0.112* (0.063)	−0.016 (0.061)	0.102 (0.065)	−0.095 (0.065)	0.056 (0.068)	−0.132*** (0.036)	−0.081 (0.050)	0.176** (0.073)
Low income	0.009 (0.032)	−0.033 (0.034)	0.063* (0.033)	0.153*** (0.035)	−0.016 (0.035)	0.071* (0.037)	−0.039* (0.023)	−0.011 (0.026)	0.171*** (0.042)
Low income × Quartile 2	−0.021 (0.044)	−0.014 (0.046)	−0.019 (0.044)	0.023 (0.047)	0.008 (0.049)	−0.122** (0.050)	0.035 (0.030)	0.006 (0.035)	−0.131** (0.056)
Low income × Quartile 3	0.048 (0.045)	0.031 (0.046)	−0.030 (0.046)	0.013 (0.048)	−0.003 (0.049)	−0.047 (0.050)	0.048** (0.027)	−0.016 (0.036)	−0.113** (0.054)
Low income × Quartile 4	0.059 (0.047)	0.005 (0.049)	−0.056 (0.047)	0.018 (0.050)	0.055 (0.051)	−0.022 (0.052)	0.097*** (0.030)	0.002 (0.038)	−0.078 (0.054)
(Low Inc) + (Low Inc *Q2)	−0.011 (0.031)	−0.048 (0.032)	0.043 (0.031)	0.176*** (0.033)	−0.008 (0.034)	−0.050 (0.035)	−0.037 (0.018)	−0.005 (0.024)	0.040 (0.039)
(Low Inc) + (Low Inc *Q3)	0.057* (0.033)	−0.002 (0.032)	0.033 (0.032)	0.140*** (0.033)	−0.018 (0.035)	0.025 (0.036)	0.009 (0.016)	−0.027 (0.024)	0.058* (0.035)
(Low Inc) + (Low Inc *Q4)	0.068** (0.035)	−0.029 (0.036)	0.006 (0.035	0.135*** (0.036)	0.040 (0.038)	0.049 (0.038)	0.058*** (0.020)	−0.008 (0.027)	0.093*** (0.036)
Number of students	2,831	2,831	2,831	2,831	2,831	2,831	2,831	2,831	2,112
R2	0.064	0.180	0.113	0.115	0.106	0.118	0.030	0.045	0.078

Notes: The table reports individual OLS on the low-income dummy, quartile dummies, and interactions between the two from a regression where the outcome is indicated in column headings. I estimate Eq. (4) in the main text. The bottom quartile is the omitted quartile. All regressions include a female, race and ethnicity indicators, a USA born indicator, a first-generation student indicator, a transfer-student indicator, an ESL-learner indicator, class-level indicators (including a graduate student indicator), major dummies, and Fall 2019 cumulative GPA. Robust standard errors are reported in parentheses.

*, **,*** Estimate significantly different from zero at the 0.1 or 0.05 level or 0.01 level.

Lower-income students worked, on average, 3.9 percentage points less due to the pandemic but received 15 percentage points more emergency relief funds than their higher-income peers. Both effects are statistically significant at the 5% or lower (estimates available from author upon request). While there is no differential effect by pre-pandemic performance for receipt of emergency relief funds among lower-income students, the income differential effect on working less in Spring 2020 is driven by the lowest performing students (albeit only marginally significant). This suggests that a plausible driver of the differential Spring 2020 GPA prior to exercising the flexible grading between lower- and higher-income students in the bottom quartile could be the post-COVID-19 lower opportunity cost of studying among lower-income students.

It is noteworthy that, because of the pandemic, top-performing lower-income students were 9.7 percentage points more likely to report asking for an incomplete than their bottom-performing lower-income peers and 5.8 percentage points more likely to ask for an incomplete than their higher-income peers as seen in column 7 of Table 4. Both estimates are significant at the 1% level. It is important to underscore that the students’ survey responses to the influence of the flexible grading policy is not directly comparable to the observed in the transcripts because they capture a change in behavior relative to the pandemic. To the extent that incompletes are allowed under normal times and students can sometimes also select Pass/No Credit option, the survey responses may be picking up a change whereas the transcript data only measure levels for Spring 2020.

In contrast, bottom-performing lower-income students were 7.1 percentage points more likely to report choosing a pass/fail or NC grade than their higher-income peers as seen in column 6.8 Even though, this estimate is only marginally significant (at the 10% level), it corroborates results from transcript data suggesting that the bottom-performing lower-income students improved their Spring 2020 academic performance by converting poor grades in pass/fail, preventing them from lowering the semester GPA and dropping courses.

It is interesting to underscore that the bottom-performing lower-income students are 17.1 percentage points more likely than their higher-income peers to report facing challenges to maintain financial aid. To maintain financial aid (including the Pell grant), students must be enrolled in at least 6 credits (12 to receive the full amount) and maintain a GPA of 2.0. Hence, it is not surprising that lower-income bottom-performing students, concerned with losing their Pell Grant or having to return a portion of the funding already received, may had been willing to use the pass/fail grade to avoid getting a grade that would have hurt their GPA.

Interestingly, there is a statistically significant income differential in students’ challenges to maintain financial aid for those in the third and top quartile, although this differential is half to two thirds of the size observed among bottom performers. This income differential among top performers may reflect their need to maintain a certain GPA to preserve merit-based scholarship, which would explain their self-reported higher intake of incompletes due to COVID-19.

Conclusion

Using individual students’ administrative academic records from Spring 2017 to Spring 2020 and controlling for individual fixed effects, this paper first documents an increase in college students’ Spring 2020 GPA relative to prior academic performance. Difference-in-differences models and event-study analyses with individual fixed effects reveal heterogeneity by pre-pandemic income and pre-pandemic academic performance inequalities.

I find that bottom-performing lower-income students outperform their higher-income peers. This differential is fully explained by students’ use of the flexible grading policy with lower-income students being 35% more likely to exercise the pass/fail option than their counterparts. Students’ survey responses suggest that lower-income students’ greater concerns with maintaining financial aid may be behind their higher use of flexible grading.

In contrast, there is no GPA advantage by income among top-performing students. Instead, in the absence of the flexible grading policy, lower-income top-performing students would have underperformed relative to their higher-income counterparts. I find suggestive evidence that this lower performance may be driven by lower-income top-performing students experiencing greater challenges with online learning than their wealthier peers. These students also reported a higher use of incompletes than their higher-income peers and being more concerned about maintaining (merit-based) financial aid. Transcript data also reveals that they were less likely to exercise the pass/fail option than bottom-performing lower-income students relative to the differential observed between their wealthier counterparts.

The current research underscores the relevance of the flexible grading policy and its differential use based on students’ particular needs. The findings in this paper suggest that the flexible grading policy was able to counteract negative shocks, especially among the most disadvantaged students. Because low-income students regularly face idiosyncratic challenges, the results in this paper suggest that a higher use of the pass/fail grade (if not for all courses, for certain courses) may support students during critical moments. Future research ought to analyze how the pandemic will affect students’ academic performance in the medium and longer run, especially once the flexible grading policy no longer applies and in-person teaching resumes. Another interesting line of research is to analyze whether flexible grading per se will impact students’ future academic performance.