Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico.

There is evidence of learning loss due to the COVID-19 pandemic, but there is no related information for Mexico. The objective of the study is to estimate learning loss in reading and numeracy in Mexico. We compared the results of two household surveys conducted in 2019 and 2021. We interviewed 3161 children between 10 and 15 years. We estimated a learning loss according with SES in a range from 0.34-0.45 SD in reading and 0.62-0.82 SD in mathematics by COVID-19 pandemic, and an increase in learning poverty in a range of 25.7%-15.4% in reading and 29.8%-28.8% in numeracy. Gaps in fundamental learning by gender and SES increased. There is an urgent need develop a clear strategy to perform personalized diagnoses and implement remedial courses to address learning loss.

Introduction

The COVID-19 pandemic, and the associated school closures, have represented one of the most critical educational emergencies of the last 100 years. As we will see below, evidence has been generated regarding the adverse effects prolonged school closures had on fundamental learning. This information is critical for education systems to develop strategies to help children recover from lost learning. For this reason, it is necessary to know in greater detail how much has been lost and how far behind children will be when they return to school.

In this sense, the research problem is the lack of information on the effects of the COVID-19 pandemic on fundamental learning in reading and arithmetic in Mexico. The closure of schools associated with the COVID-19 pandemic is related to learning loss and an increase in learning poverty. In Addition, Mexico is a country that kept its schools closed for many weeks, which has increased the educational inequality gap and deepened the school dropout crisis, making it increasingly difficult to maintain educational quality. There are simulations regarding the impact of this closure on learning, but there is no information based on empirical evidence.

COVID-19, learning loss and learning poverty

The COVID-19 pandemic has had very adverse effects on education, with school closures that have affected more than 190 million students worldwide (UNESCO, 2021). The regions most affected by the pandemic in terms of health and education are Latin America and the Caribbean (Rieble-Aubourg and Viteri, 2020, World Bank, 2021). This region has not only experienced a very high concentration of deaths concerning its population (John Hopkins University. 2021) but has kept schools closed longer than many other regions (UNESCO, 2021).

As has been widely analyzed, school closures and health and economic crises increase the inequality gap, the risk of dropout, educational exclusion, and, as we will see in this article, learning loss (Fore, 2021, GEM-UNESCO, 2020, United Nations, 2020, World Bank, 2020) and learning poverty (Azevedo et al., 2020, Azevedo and Montoya, 2021, World Bank, 2019a).

Learning loss can be defined as “any specific or general loss of knowledge and skills or reversals in academic progress, most commonly due to extended gaps or discontinuities in a student’s education” (Huong et al., 2020, p. sp). Before the pandemic, learning loss was associated with summer vacations, when a drop in academic performance and a widening achievement gap among students were observed (Kuhfeld, 2019). However, with the pandemic, interest in measuring learning loss has increased, intending to reduce learning loss and inequalities in learning and dropout (Azevedo et al., 2020, Donnely and Patrinos, 2021, Huong et al., 2020, Iqbal et al., 2020, Kaffenberger, 2020, Pier et al., 2021).

At the beginning of the pandemic, statistical estimates were made to predict these losses. For example, Iqbal et al. (2020) estimated a loss of between 0.3 and 0.9 quality-adjusted years of schooling according to data from 157 countries. Azevedo et al. (2020), through a simulation, estimated that after five months of school closure due to COVID-19, there could be a loss of 0.6 quality-adjusted years of schooling, an increase of 25% in “insufficient” level scores on standardized tests, and an increase in learning poverty, especially in the poorest countries.

These projections have been complemented with empirical analyses in developed countries, which have estimated the learning loss in various contexts, including basic and higher education. Donnely and Patrinos (2021) identified studies that show learning loss in at least some of the students analyzed. In basic education, Maldonado and Witte (2021) identified losses of 0.19 SD in mathematics in Belgium and 0.29 SD in Denmark for 6th graders in elementary school. Engzell et al. (2020) analyzed the learning loss in mathematics, reading, and writing in the Netherlands in boys and girls between eight and 11 years old and estimated a loss of 0.08 SD between 4th and 6th grade in these domains. Kuhfeld et al. (2020), in a sample of 4.4 million students aged three to eight years in the United States, estimated losses in mathematics of 5–10% points, with no differences in reading, using an at-home online test (MAP Growth Assessment). Gore et al. (2021) analyzed the closure of schools in Australia and found that after 8–10 weeks of closure, there were losses in mathematics and reading in three elementary schools. The study by Schult et al. (2021) of 5th graders in Germany found that after 8.5 weeks of closure, there was a loss of 0.03–0.09 SD in mathematics and 0.07 SD in reading. Only one study, by Tomasik et al. (2020), found no significant differences in mathematics in a sample of 25,685 students in Switzerland after eight weeks of school closure (Donnely and Patrinos, 2021, p. 149).

These empirical analyses have been published only for high-income countries with a relatively short closure of schools. Thus, except for the simulations, there are no estimates of the learning loss in middle-income countries or with more extended school closings. For this reason, the first objective of this article is to estimate learning losses due to the COVID-19 pandemic in southern Mexico.

Learning poverty

On the other hand, just before the pandemic, the UNESCO Institute for Statistics and the World Bank coined the concept of learning poverty. Learning poverty is the inability to read and understand a simple text by ten years. This indicator combines schooling and learning indicators: it begins with the proportion of children who have not achieved minimum reading proficiency (as measured in schools) and is adjusted by the proportion of children who are not in school (and are assumed to be unable to read proficiently) (World Bank, 2019a, p. 6)(World Bank, 2019b).

This concept allows the inclusion of some of the results of the Citizen-Led Assessment, which was developed in the global south to generate evidence on the difference between schooling and learning (schooling is not learning) (Pritchett, 2013). Additionally, it allows an understanding of the intensity of the learning crisis in the countries of the global south (Banerji, 2013, Banerji et al., 2013, Hevia and Vergara-Lope, 2020, Munene, 2016) and is a pioneering home-based assessment for investigating the learning of all children, whether they attend a school or not (Banerji, 2017, Mugo et al., 2015, PAL, 2018, Ruto, 2013, Wilson, 2018). The Independent Learning Measurement Project (MIA, for its acronym in Spanish), which conducts Citizen-Led Assessments in southern Mexico, generated information on the learning lag between 2015 and 2019 for various states in southeastern Mexico (Hevia and Vergara-Lope, 2019, Vergara-Lope et al., 2017, Vergara-Lope and Hevia, 2016). However, there is no information on the effects of school closures on learning poverty in Mexico. Therefore, the second objective of this article is to estimate the learning poverty in southern Mexico.

The COVID-19 pandemic in Mexico

The effects of COVID-19 in Mexico have been profound in health, economics, and education, as we will review in this document. Regarding health, a total of 326,612 excess deaths were recorded in 2020 (Mejía et al., 2021), and as of May 2021, there were 223,068 deaths. In the economic sphere, the GDP fell by 8.5% (INEGI, 2021a).

Regarding education, between March 2020 and May 2021, schools were closed for 48 weeks. According to the National Institute of Statistics and Geography (Instituto Nacional de Estadística y Geografía - INEGI), in the 2019–2020 school year, 740,000 students between 3 and 29 years old (2.2%) did not finish the school year, 58.9% for reasons associated with COVID-19% and 8.9% for lack of money or resources. Of these, 28.8% dropped out of school because they had lost contact with their teachers or could not do homework, 22.4% dropped out because someone in the household was left without a job or had a reduced income, and 17.7% left because they lacked a computer, other device, or Internet connection (INEGI, 2021b, p. 12). Therefore, for the 2020–2021 school year, 5.2 million students between 3 and 29 years old (9.6%) were not enrolled in school due to the COVID-19 pandemic and a lack of economic resources (INEGI, 2021b). This situation is similar to the estimates by the Secretariat of Public Education (SEP, for its acronym in Spanish), which, at the beginning of the 2020–2021 school year, indicated that approximately 10% of students at the primary level and 8% at the high school level had dropped out (Arellano, 2020).

The “Learn at home” strategy continued to deliver its original program and study plan by remote means, initially via the internet and, during the 2020–2021 school year, via television (SEP, 2020b). According to a telephone survey conducted by the Research Institute for Development with Equality (Instituto de Investigaciones para el Desarrollo con Equidad - EQUIDE), in May 2020, only 60% of households reported having seen or heard about this program, and the proportion decreased according to socioeconomic status (SES) (Pérez and Gaitan, 2020, pp. 3–4). Despite this, according to the government, this strategy allowed eight out of 10 students to establish constant communication with their teachers during the 2019–2020 cycle and nine out of 10 students to “maintain their learning” (SEP, 2020a). This last statement, however, was only supported by an opinion poll. As of September 2021, no document or research would allow an estimation of the effect of the pandemic on learning. For these reasons, the objectives of this article are to estimate the learning loss and learning poverty in Mexico.

Methodology

To meet the objectives of this research, the results of two household surveys that used the same tools to measure fundamental learning are compared. The first was developed by the MIA Project and conducted in November 2018 in Yucatan, Quintana Roo, and Campeche, and its results were published in February 2019 (MIA, 2019). In this survey, 3438 individuals between 7 and 17 were interviewed, and the MIA tool was applied to measure reading and basic mathematics (Hevia and Vergara-Lope, 2016). The second survey was implemented by the Center for Educational and Social Studies (Centro de Estudios Educativos y Sociales - CEES) and Mexicanos Primero in May 2021 in the states of Campeche and Yucatan. In Addition, in this survey, 2000 individuals between 10 and 15 were interviewed, and a questionnaire adapted from MIA that included the same items that were applied in 2019.

Participants

For this research, only participants between 10 and 15 years old who lived in the states of Campeche and Yucatan were selected from the 2019 survey, for a total of 1172 individuals. A total of 1989 individuals participated in the 2021 survey. Thus, the sample consisted of 3161 students between 10- and 15-years old living in the states of Campeche and Yucatan, with a mean age of 12.07 years (SD = 1.63); 49.64% (N = 1569) were boys, and 50.36% (N = 1592) were girls. Both samples include children in and out of school. In the 2019 sample, 97.7% were attending, while in 2021, after 14 months of closed schools, it dropped to 92.15%.

Sample

In the 2019 survey, the total number of households in the states of Yucatan, Quintana Roo, and Campeche was considered the population universe, according to the results of the 2015 inter census survey (INEGI, 2015). Therefore, the sample design was defined with a probability of 95% (z = 1.96), a margin of error of + /- 2%, a power of 0.8, and a type I error of 0.03. Based on this calculation, a sample of 2564 homes was selected.

Multistage, probabilistic, stratified cluster sampling was performed (Hernández Sampieri et al., 2001, Kerlinger and Lee, 2002). The final unit of selection was the home, and the observation units were boys, girls, and young people between 7 and 17 years old who lived in the selected home. For the selection of homes, the sampling procedure involved four steps:

1. Selection of clusters or primary sampling units through stratified systematic sampling and with probability proportional to size, which in this case was the municipalities that make up each electoral district.

2. Selection of the secondary sampling units, which in this case were rural localities and primary geostatistical areas (Áreas Geoestadística Básica-AGEB) for urban areas.

3. Random selection of blocks within the cluster.

4. Systematic selection of homes in the blocks.

All boys and girls between 7 and 17 years old who lived in the selected households and wanted to participate in the study were interviewed. The information was collected from 2564 households in 39 urban and rural locations in 20 different municipalities. The surveys were performed by 293 volunteers who received six effective hours of prior training.

In the 2021 survey, to calculate the representative sample, the universe of observations corresponded to the list of beneficiaries of scholarships from the Benito Juárez Welfare Program (Programa Nacional de Becas para el Bienestar Benito Juárez) for the 2019 fiscal year, which was the last year for which the scholarships were available. Therefore, the sample has a probability of 95% (z = 1.96) with a margin of error of + /- 3%, a power of 0.8, and a type I error of 0.03 and included 2037 boys and girls. In Addition, the selection of municipalities and localities was proportional to size, and individuals from 20 municipalities were interviewed (CEES, 2021).

Sample-Comparability procedures

Although both surveys are random, they were designed to identify households, and all children living in that household were interviewed. The following procedures were carried out to ensure the comparability of the bases. First, a Socio-Economic Status-SES index was generated composed of the schooling of the head of household and household characteristics: number of rooms, internet connectivity, and the existence of a car. Using Pearson’s correlation, this index has a moderate and significant correlation with the Díaz-Acosta et al. (2015) proposal (0.51 sig at 0.001), and a significant and robust correlation with the Mexican Association of Marketing Research and Public Opinion (Asociación Mexicana de Agencias de Inteligencia y Mercado de Opinión; AMAI, 2018) measurement (0.86 sig at 0.001).

Second, using normal quartiles, four levels were generated: high, medium-high, medium-low, and low SES. Third, we compared means by age between 2019 and 2021 for each SES level, finding no significant differences ( Table 1). For gender we obtained no significance Chi-squared between 2019 and 2021 sample, for each SES level.

Table 1

SES groups by age and gender.

Socio-Economic Status-SES	Group	N	Age			Gender
			Mean	T	P	% men	% women	X2	p
Low SES	2019	340	12.1559	1.6797	NS	52.1	47.9	1.06	NS
	2021	648	11.9707			48.6	51.4
Middle-Low SES	2019	207	12.1643	1.6808	NS	48.3	51.7	1.396	NS
	2021	462	11.9416			53.3	46.8
Middle-High SES	2019	210	12.0571	-0.5509	NS	50.1	49.1	2.261	NS
	2021	591	12.1303			48.9	51.1
High SES	2019	336	12.2262	1.7065	NS	52.7	47.3	0.71	NS
	2021	300	12.0067			49.3	50.7

Source: Authors’ analysis.

Tools

For reading and mathematics, the MIA tool was applied (Hevia and Vergara-Lope, 2016). The tool is similar to other Citizen-Led-Assessment instruments, like ASER in Asia or UWEZO in Africa, and consists of 10 items: five for reading (Syllables, Words, Sentences, History, Comprehension) and five for mathematics (identification of Numbers, Addition, Subtraction, Division, Problem-solving). For reading, the difficulty of the instruments corresponds to 2nd grade. In mathematics, number recognition, addition and subtraction are aligned to 2nd grade and division and problems to 4th grade. To score the instrument, each item has a value of one point, so the scores range from 0 to 5 in reading and mathematics (Hevia and Vergara-Lope, 2016). The tool’s reliability was calculated using Cronbach’s alpha coefficient; internal consistency indexes of 0.82 in general, 0.78 for reading, and 0.74 for mathematics were obtained, which are considered acceptable levels (Celina Oviedo and Campo Arias, 2005). During the 2021 administration, item 4 (History) was not applied; therefore, the analyses were performed using four reading and five mathematics items.

Analytic procedures

Diverse descriptive and inferential statistical tests were used for the analysis. First, we used descriptive statics to determine fundamental learning in reading and mathematics. Second, we used Pearson correlations and the Spearman test to identify the factors associated with educational achievement. Third, means were also compared using parametric (Student’s t-test) and non-parametric test (U-Mann-Whitney), and the effect size was calculated using Cohen’s d (Ellis, 2010, Sullivan and Feinn, 2012). These procedures were performed using the statistical package Stata 14.1.

Results

The results are presented in three sections: the first shows a big picture of the learning loss and estimates its effect cording by SES; in the second, the increase in learning poverty is estimated; and in the third, comparisons are made according to gender.

Learning loss in reading and mathematics

In reading, there was a learning loss for all items and at all ages and SES groups analyzed. In this sense, early ages (10 and 11 years) and Low SES show the steepest drops. For example, at age 10, on the Comprehension item, the falls are 25% in Low SES and 15% in High SES. The accumulated results shows that the most critical reduction was on comprehension in every SES quartile (see Table 2).

Table 2

Comparison of results for each level of reading by year, according to age and SES quartiles (Percentages).

Age	Syllable		Words		Sentences		Inferential comprehension
	2019	2021	2019	2021	2019	2021	2019	2021
Low SES (N 2019 =340; N 2021 =648)
10 years old	94.3	90.2	94.3	81.7	89.8	65.2	58.0	32.3
11 years old	98.1	91.5	96.2	89.8	88.5	68.6	53.8	38.1
12 years old	98.0	94.2	96.1	89.3	92.2	80.2	66.7	47.9
13 years old	98.2	95.4	98.2	92.6	98.2	82.4	76.8	52.8
14 years old	100.0	100.0	100.0	100.0	98.1	93.5	87.0	75.0
15 years old	97.4	93.3	97.4	88.9	94.9	88.9	76.9	66.7
Mean SES low	97.4	93.7	96.8	89.5	93.2	77.2	68.5	48.1
Middle Low SES (N 2019 =207; N 2021 =462)
10 years old	97.8	95.5	95.6	89.2	93.3	73.9	44.4	43.2
11 years old	100.0	98.9	100.0	95.4	97.1	77.0	74.3	49.4
12 years old	100.0	97.1	100.0	95.1	94.9	92.2	74.4	56.3
13 years old	100.0	100.0	97.3	98.5	97.3	96.9	83.8	73.8
14 years old	100.0	98.6	100.0	97.2	96.2	91.5	84.6	60.6
15 yerars old	95.8	96.0	95.8	96.0	95.8	92.0	87.5	68.0
Mean SES middle-low	99.0	97.6	98.1	94.6	95.6	85.5	72.3	55.6
Middle High SES (N 2019 =210; N 2021 =591)
10 years old	97.6	95.4	97.6	87.7	95.2	76.2	76.2	51.5
11 years old	100.0	95.9	100.0	94.3	96.1	84.4	72.5	53.3
12 years old	100.0	95.6	100.0	94.5	95.1	90.1	82.9	64.8
13 years old	100.0	100.0	100.0	100.0	100.0	97.9	85.7	75.8
14 years old	100.0	98.9	100.0	97.9	96.0	96.8	88.0	76.8
15 yerars old	100.0	96.6	100.0	96.6	100.0	93.1	91.3	75.9
Mean SES Middle high	99.5	97.0	99.5	94.6	96.7	88.5	81.0	64.3
High SES (N 2019 =336; N 2021 =300)
10 years old	98.5	95.8	98.5	93.1	98.5	84.7	72.3	56.9
11 years old	100.0	98.2	100.0	96.5	98.2	87.7	80.7	68.4
12 years old	100.0	100.0	100.0	98.2	97.5	94.5	81.0	67.3
13 years old	100.0	97.9	100.0	97.9	100.0	95.7	88.0	70.2
14 years old	100.0	100.0	97.6	100.0	97.6	98.0	81.0	86.3
15 yerars old	100.0	100.0	100.0	100.0	100.0	100.0	93.0	83.3
Mean SES high	99.7	98.3	99.4	97.0	98.5	92.0	81.8	69.7
Total (N 2019 =1093; N 2021 =2001)
Mean results	98.9	96.3	98.2	93.3	95.6	84.7	75.3	57.9

Source: Authors’ analysis.

When we compare reading means, we can first observe a learning loss in all socio-economic levels. And secondly, the losses are largest at the lowest SES ( Fig. 1).

Fig. 1

Comparison of the reading sum means between 2019 and 2021 by SES level.

The Table 3 shows a comparison of groups in reading, using the Mann-Whitney test for non-parametric data. There were statistically significant differences between 2019 and 2021 in all the SES levels. Using d-Cohen, we estimate ranging from 0.45 SD at Low SES to 0.34 SD at High SES in reading (see Table 3).

Table 3

Comparison of reading effect size between 2019 and 2021 by SES level.

SES quartile	Groups	Obs	Mean	Diff	z	d-Cohen	95% conf
Low SES	2019	340	3.56	0.4739***	6.881	0.4503	0.3175	0.5830
	2021	648	3.08
Middle-Low	2019	206	3.65	0.3172***	4.456	0.3677	0.2022	0.5329
	2021	462	3.33
Middle-High	2019	210	3.77	0.3234***	4.694	0.3797	0.2210	0.5381
	2021	591	3.44
High SES	2019	336	3.79	0.2246***	3.864	0.3466	0.1896	0.5033
	2021	300	3.57

Note: *** sig. > 0.001. Source: Authors’ analysis.

For mathematics, a similar situation occurred. Losses occurred in all items, at all ages, and all SES levels. The most significant losses were in subtraction and division skills, mainly among low SES, with more than 33 points in Subtraction and 28 points in Division. On average, we observed losses of 16.4% in Addition, 26.9% in Subtraction, 27.5% in Division, and 16.9% in Problem (See Table 4).

Table 4

Comparison of results for each level of mathematics by year, according to age and SES quartile.

Age	Numbers		Addition		Subtraction		Division		Problem
	2019	2021	2019	2021	2019	2021	2019	2021	2019	2021
Low SES (N 2019 =340; N 2021 =648)
10 Years old	96.6	92.7	87.5	59.1	67.0	20.1	34.1	4.3	14.8	0.0
11 Years old	96.2	97.5	86.5	63.6	61.5	32.2	36.5	12.7	11.5	1.7
12 Years old	98.0	95.9	88.2	71.9	66.7	36.4	39.2	17.4	17.6	5.8
13 Years old	98.2	98.1	89.3	69.4	55.4	37.0	41.1	18.5	23.2	9.3
14 Years old	100.0	100.0	96.3	77.2	74.1	40.2	53.7	22.8	35.2	10.9
15 Years old	100.0	95.6	94.9	73.3	79.5	48.9	66.7	26.7	48.7	13.3
Mean SES low	97.9	96.3	90.0	67.6	66.8	33.0	43.2	14.8	23.2	5.4
Middle Low SES (N 2019 =207; N 2021 =462)
10 Years old	100.0	92.8	84.4	65.8	55.6	36.0	26.7	11.7	4.4	5.4
11 Years old	100.0	98.9	91.4	72.4	51.4	39.1	31.4	12.6	8.6	5.7
12 Years old	100.0	97.1	100.0	75.7	64.1	44.7	46.2	27.2	23.1	3.9
13 Years old	97.3	98.5	94.6	80.0	62.2	41.5	45.9	20.0	32.4	6.2
14 Years old	100.0	97.2	92.3	76.1	88.5	46.5	80.8	25.4	38.5	9.9
15 Years old	95.8	96.0	87.5	76.0	70.8	44.0	50.0	24.0	29.2	12.0
Mean SES middle-low	99.0	96.5	91.7	73.4	63.6	41.3	44.2	19.3	20.9	6.3
Middle High SES (N 2019 =210; N 2021 =591)
10 Years old	97.6	90.8	85.7	66.9	66.7	37.7	35.7	11.5	16.7	1.5
11 Years old	98.0	96.7	88.2	74.6	70.6	28.7	45.1	13.1	23.5	7.4
12 Years old	97.6	97.8	87.8	78.0	78.0	48.4	56.1	16.5	17.1	7.7
13 Years old	100.0	98.9	92.9	85.3	71.4	62.1	50.0	40.0	17.9	16.8
14 Years old	100.0	96.8	92.0	83.2	76.0	49.5	68.0	30.5	44.0	12.6
15 Years old	95.7	100.0	87.0	81.0	69.6	55.2	60.9	32.8	47.8	24.1
Mean SES Middle high	98.1	96.3	88.6	77.2	71.9	45.0	50.5	22.3	25.2	10.2
High SES (N 2019 =336; N 2021 =300)
10 Years old	96.9	95.8	93.8	70.8	61.5	37.5	38.5	9.7	13.8	5.6
11 Years old	100.0	96.5	94.7	71.9	77.2	47.4	57.9	29.8	35.1	5.3
12 Years old	97.5	98.2	97.5	85.5	86.1	52.7	63.3	23.6	26.6	5.5
13 Years old	100.0	97.9	92.0	89.4	74.0	57.4	60.0	36.2	32.0	12.8
14 Years old	97.6	100.0	90.5	90.2	73.8	64.7	59.5	37.3	28.6	13.7
15 Years old	100.0	100.0	97.7	88.9	83.7	61.1	58.1	50.0	46.5	22.2
Mean SES high	98.5	97.7	94.6	81.0	76.2	51.3	56.0	27.3	29.2	9.0
Total (N 2019 =1093; N 2021 =2001)
Mean results	98.4	96.6	91.5	73.8	70.2	41.2	48.7	19.9	24.9	7.5

Source: Authors’ analysis.

When we compare mathematic means, we can first observe a learning loss in all socio-economic levels. And secondly, the losses are higher at the lowest SES ( Fig. 2).

Fig. 2

Comparison of the mathematics sum means between 2019 and 2021 by SES level.

The Table 5 shows a comparison of groups in mathematics, using the Mann-Whitney test for non-parametric data. There were statistically significant differences between 2019 and 2021 in all the SES levels. Using d-Cohen, we estimate ranging from 0.83 SD at Low SES to 0.62 SD at High SES in mathematics (see Table 5).

Table 5

Comparison of mathematic effect size between 2019 and 2021 by SES level.

SES quartile	Groups	Obs	Mean	Diff	z	d-Cohen	95% conf
Low SES	2019	340	3.2118	1.0405***	11.4120	0.8275	0.6911	0.9636
	2021	648	2.1713
Middle-Low	2019	206	3.1942	0.8262***	7.2910	0.6550	0.4868	0.8227
	2021	462	2.3680
Middle-High	2019	210	3.3429	0.8336***	7.5370	0.6295	0.4688	0.7897
	2021	591	2.5093
High SES	2019	336	3.5446	0.8813***	8.3610	0.6947	0.5341	0.8548
	2021	300	2.6633

Note: *** sig. > 0.001. Source: Authors’ analysis.

Increase in learning poverty

Learning poverty in reading is defined as the inability of a 10-year-old child to read and understand a text adequately (World Bank, 2019a, p. 6). From Fig. 3, three observations can be made: 1) there was a significant increase in learning poverty in all SES quartiles. As Fig. 3 shows, in Low SES, learning poverty increased to 25.7%. In High SES, there was an increase of 15.4% between 2019 and 2021; 2) learning poverty in reading and the differences between in 2019 and 2021 decreased but did not disappear with age: even in 15 years old there was a significant proportion of children who could not read or understand a simple text. For example, in 15 years, increase 10% in both Low SES and High SES, and 33.3% in Low SES and 16.7% in High SES were in a situation of “learning poverty” in 2021; and 3) there was already a significant percentage of poverty in fundamental learning before the pandemic, it increased in 2021 and both, the percentages and the increments are very related with SES conditions. Data from 2019 show gaps in learning poverty between all SES quartiles and the highest learning poverty is in Low SES in 2021.

Fig. 3

Comparison of percentage of individuals (10 and 15 years old) with learning poverty in reading between 2019 and 2021, by SES levels.

Although the World Bank does not define poverty of numeracy learning, for this article, we can define it as the percentage of 10-year-old children who cannot solve three-digit division problems without remainders (for example, 210/3). Using this definition, learning numeracy poverty in the analyzed population was ranged from 88.3% to 95.7%, according to socioeconomic quartile. When comparing 2019 and 2021, we see that math learning poverty grew 29.9 points in Low SES, 15% in Middle-Low SES, 24.2% in Middle-High SES, and 28.8% in High SES. Similarly with reading poverty, numeracy poverty decreases with age but at a lower rate. ( Fig. 4).

Fig. 4

Comparison of percentage of individuals (10 and 15 years old) with learning poverty in mathematics between 2019 and 2021, by SES levels.

Differences by gender

Finally, this section presents the comparisons between groups according to gender. The gaps between 2019 and 2021 are compared by gender according to with SES quartiles. As Table 6 shows, in Low SES, both boys and girls presented significant decreases in learning. In mathematics the losses are more unequal, in Low SES girls lost more (0.93 SD), while in High SES boys lost more (0.72 SD) (Table 6).

Table 6

Comparison of reading and mathematics effect size between 2019 and 2021 by gender and SES level.

		Gender	Year	N	Means	Diff.	z	d-Cohen	95% conf
Low SES	Reading	Girls	2019	163	3.503	0.500***	4.875	0.450	0.260	0.639
			2021	333	3.003
		Boys	2019	177	3.610	0.439***	4.790	0.445	0.258	0.631
			2021	315	3.171
	Math	Girls	2019	163	3.319	1.181***	8.922	0.931	0.734	1.127
			2021	333	2.138
		Boys	2019	177	3.113	0.907***	7.203	0.728	0.538	0.918
			2021	315	2.206
High SES	Reading	Girls	2019	159	3.774	0.260***	2.926	0.358	0.133	0.581
			2021	152	3.513
		Boys	2019	177	3.814	0.185*	2.488	0.330	0.110	0.550
			2021	148	3.628
	Math	Girls	2019	159	3.358	0.845***	5.597	0.663	0.434	0.891
			2021	152	2.513
		Boys	2019	177	3.712	0.894***	6.203	0.719	0.493	0.943
			2021	148	2.818

Note: *** sig. > 0.001; * sig. > 0.05. Source: Authors’ analysis.

Conclusion

The results are shown here reflect an educational emergency because of the COVID-19 pandemic and the closure of schools for more than 48 weeks in Mexico. The effects of the pandemic have manifested in a widespread way in both genders, in all the ages studied and for children with different SES.

Although there were already severe lags in fundamental learning in reading and mathematics (2019 measurement), the loss that occurred after more than 12 months of school closures was evident (2021 measurement), and much more severe in the low SES level. The results regarding the learning loss and the increase in learning poverty in reading and mathematics are consistent with both empirical analyses conducted in other parts of the world (Donnely and Patrinos, 2021) and existing estimates and simulations (Azevedo et al., 2020).

These results also show a trend similar to the Comprehensive Learning Assessment (Diagnóstico Integral de los Aprendizajes) conducted by the government of Chile. Furthermore, this study shows that 43% of students are below the expected level in mathematics, and 60% are below the expected level in reading (Agencia de Calidad de la Educación, 2021), confirming that the learning loss due to school closures is a generalized phenomenon.

The magnitude of the learning loss forces governments and society in general to generate an urgent strategy of return to classes that has two priorities: the generation of diagnostic methods, training, and personalized evaluations that give teachers as specific a picture as possible of the diversity of learning levels that they will find when their students return to school. For this purpose, valid and reliable tools, such as those of the MIA Project, can be used to obtain immediate information regarding the reading and mathematics levels of each child with a few minutes of application; such information can be used to make decisions about the needs of a particular child and implement a leveling strategy based on the child’s learning and not on his or her grade in school.

Second, remedial, and leveling courses should be designed and implemented to address this learning loss, as recommended by the United Nations (2020) and prominent international organizations. The “Teaching at the Right Level” (TaRL) strategy seems to be particularly relevant and effective at both the international level (Banerjee, 2016) and in Latin American, where the MIA project has been able to evaluate remedial courses that show positive and significant effects for addressing this lag (Hevia and Vergara-Lope, 2021).

The implementation of remedial courses implies difficulties guaranteeing minimum infrastructure conditions and adequate health safety protocols in each school. In Addition, it is necessary working on comprehensive recovery that does not neglect the socioemotional and mental health situation of teachers, families, and the children themselves.

Comparative experience recommends incorporating modifications to the curriculum to provide greater flexibility to teachers, avoid content saturation and incorporate formative assessment for high-impact decisions, such as student promotion. A recent report from the region highlights that, among the fifteen countries participating in the Latin American Laboratory for Assessment of the Quality of Education, only Brazil, Peru, and Mexico have not modified their curricula in this sense (UNESCO/OREALC, 2021).

In sum, in a comprehensive strategy of recovery, reinforcement, and leveling, diagnostic assessment is the necessary step to identify and remedy setbacks in equity and quality of education and to guide specific, non-homogeneous interventions required to achieve the fundamental learning that was not achieved (UNESCO/OREALC, 2021).

However, the learning lag is not a problem that can be solved in a few weeks or with the use of leveling courses alone; in this sense, together with an emerging strategy, we agree with the United Nations (2020) that there is a need to develop fairer and more egalitarian educational systems that leave no one behind.

Funding

This work was supported by the [grant 2018-6968].

Author statement

Conception and design of Study: Felipe J. Hevia, Samana Vergara-Lope. Acquisition of data: Anabel Velásquez-Durán; David Calderón. Analysis and interpretation of data: Felipe J. Hevia, Samana Vergara-Lope. Drafting the manuscript: Felipe J. Hevia, Samana Vergara-Lope. Revising the manuscript critically for important intellectual content: Anabel Velásquez-Duran, David Calderón. Approval of the version of the manuscript to be published: Felipe J. Hevia, Samana Vergara-Lope, Anabel Velásquez-Durán, David Calderón.