No learning loss in Sweden during the pandemic evidence from primary school reading assessments.

The COVID-19 pandemic has led to worldwide school closures, with a risk of learning loss. Sweden kept primary schools open, but it is unknown whether student and teacher absence and pandemic-related stress factors affected teaching and student progress negatively. In this study, reading assessment data from 97,073 Swedish primary school students (grades 1-3) were analysed to investigate potential learning loss. Results showed that word decoding and reading comprehension scores were not lower during the pandemic compared to before the pandemic, that students from low socio-economic backgrounds were not especially affected, and that the proportion of students with weak decoding skills did not increase during the pandemic. Study limitations are discussed. We conclude that open schools benefitted Swedish primary school students.

Introduction

Since March 2020, the COVID-19 pandemic has had a great impact on all aspects of society globally, but different countries have chosen different strategies when trying to limit the spread of the coronavirus. School closures affected over 90% of the world's students early in the pandemic (UNESCO, 2020), and one year later almost half of the world's students were still affected by partial or full school closures (UNESCO, 2021b). UNESCO (2021a) lists many negative consequences of school closures, including interrupted learning, and research shows that this is especially true for the younger and the most disadvantaged students (Alban Conto et al., 2021; Azevedo et al., 2020), who might have learning challenges and/or not have access to e.g., tools and internet access for online learning or help from caregivers. Even without school closures, however, the pandemic might affect student well-being and learning. This study aims to compare reading skills before and during the pandemic in primary school children (grades 1-3) in Sweden, a country that chose to keep preschools, primary schools, and lower secondary schools (i.e., educational settings for children ages 1-15) open during the COVID-19 pandemic.

Early models of effects on school closures during the pandemic (for the 2019-2020 school year) showed rather grim projections (Azevedo et al., 2020; Bao et al., 2020; Kuhfeld et al., 2020). Kuhfeld et al. (2020) used estimates from the absenteeism literature as well as summer learning loss data and projected that the negative effects on reading for US students would correspond to starting school in fall 2020 with 63-68% of the typical learning gains. Bao et al. (2020) used previous longitudinal data on reading from US children in kindergarten and first-grade children to model reading growth during the summer months when schools are closed, to then estimate the effects of school closures during the pandemic. Similar to Kuhfeld et al. (2020), their models showed that students in grade 1 would start the fall semester of 2020 with a 31% less reading ability gain than usual.

In 2020-2021, real-world data on learning during the pandemic from countries where students have experienced school closures have started to get published (for a systematic review including studies published up until April 31, 2021, see Hammerstein et al., 2021). The studies included in Hammerstein et al. (2021) are from different countries (all from the global north including Australia, except for one study from China) investigating the effect of 7-8 week school closures, and report mixed findings regarding loss in reading attainment in primary grades, with effect sizes ranging from -0.37 SD (Switzerland: Tomasik, Helbling, & Moser, 2021) to +0.04 SD (Australia: Gore et al., 2021): i.e. ranging from the worst-case projections to no negative effect at all). The magnitude of effect on reading due to school closures are dependent on what other measures are in place, such as good access to and knowledge of digital and online solutions, economic support for families during the pandemic, etc. and it is therefore not straightforward to compare results between countries. A few studies have confirmed that school closures did have a particularly negative effect on disadvantaged students (Engzell, Frey, & Verhagen, 2021; Maldonado & De Witte, 2021), low-achieving students (Clark et al., 2021), and younger students (Tomasik et al., 2021).

Sweden made the choice to keep pre-schools, primary schools, and lower secondary schools open very early in the pandemic, and even when upper secondary schools and universities closed and went online, schools for the younger students have been kept in-person throughout the whole COVID-19 pandemic. This choice was in stark contrast to most other comparable countries, including the close Nordic neighbours. Open schools did not necessarily mean that the pandemic did not affect the learning of the youngest students in Sweden, however. The Swedish pandemic strategy included strict recommendations to stay home if the slightest symptoms of illness were present. Increased teacher and student absence and the resulting difficulty with continuity in teaching and learning, as well as the anxiety and stress from experiencing a pandemic might still have affected teaching as well as student learning negatively (Sjögren et al., 2021). One group of students that might have been disproportionally affected are those who need more intense and specialized support in school, such as those with weaker reading abilities or slow reading development. Another group is students from disadvantaged socio-economic backgrounds. A lower socioeconomic status and/or being a first-generation immigrant from a low- or middle-income country are associated with a higher risk of getting infected with and dying from COVID-19 in Sweden (Drefahl et al., 2020). Drefahl et al. (2020) point out that explanations might include both residential and occupational risk factors, e.g., multigenerational households and no option to work remotely. Thus, Swedish children from lower-socioeconomic backgrounds had a higher risk of being infected with COVID-19 themselves, and also an increased risk of other pandemic-related stress factors, such as sickness and death in the family, more crowding and potential conflicts at home, and less protective social contacts outside the family (Sjögren et al., 2021).

If, and how, the pandemic has affected student learning in Sweden is unknown, however. In many countries, educational data have not been collected and shared during the pandemic. In Sweden national tests in reading and mathematics in grade 3 were cancelled in spring 2020, and in spring 2021, schools did not have to report the results to Statistics Sweden (SCB), to ease the administrative burden on schools. Thus, no official national data on student progress in reading during the pandemic compared to previous years exist.

LegiLexi (www.legilexi.org) is a nonprofit organization with a mission to teach all Swedish students to read. LegiLexi provides evidence-based support on early literacy instruction to teachers via an online portal and an e-book, and an online assessment tool developed by researchers to enable a forward-looking view on students’ development in pre-reading, oral language, and literacy skills, all provided with no cost. The assessment tool currently targets teachers in grades 1-3 and includes subtests in letter knowledge, phonological awareness, vocabulary, listening comprehension, single word decoding, nonsense word decoding, and reading comprehension. Teachers are recommended to use the assessment tool during three test periods each academic year (with the suggestion to test in September (fall), December/January (winter), and May (spring)), but there are no requirements that all students are always tested with all subtests. The online assessment tool was released in 2017, and in November 2021, the tool reached 26 000 registered teachers from all 290 Swedish municipalities (the local government entity responsible for community services, including e.g., schools and care for the elderly). Anonymized data from all online test sessions are kept by LegiLexi and are used to investigate the effects of better literacy instruction, and for research. LegiLexi data from before and during the COVID-19 pandemic are the basis of the current study, and scores from two of the subtests that most directly reflect reading ability, decoding of single words, and reading comprehension, were analysed.

Given that Sweden took a different path during the pandemic than many other countries, it is valuable to see if and how the pandemic has affected reading development in the youngest student groups and particularly if the reading skills of Swedish students with lower reading skills, or those from more disadvantaged backgrounds have been disproportionally affected. Data on reading skills in a large cohort of Swedish students in grades 1-3 from the years before the pandemic (fall 2017 to winter 2019/2020) and during the pandemic (spring 2020 to spring 2021) are presented. Both cross-sectional data with comparisons between academic years and each test period (to include as many students as possible), and longitudinal data for the sub-groups of students where data exists for all three test periods across one grade were analysed, to respond to the following research questions: 1) Was there a negative effect on Swedish students’ early reading development (decoding and reading comprehension) during the first two waves of the COVID-19 pandemic (March 2020 to June 2021) in grade 1, 2 and 3 respectively? Specifically, a) Were reading scores lower during the COVID-19 pandemic compared to before the pandemic? and b) Did the proportion of students with weak reading skills in Sweden increase during the COVID-19 pandemic? 2) Was there a negative effect on early reading development for the subset of Swedish students in schools serving a large proportion of low socio-economic status households during the COVID-19 pandemic? (The same sub-questions as in question 1 were addressed.)

Materials and methods

According to Swedish law (2003:460), ethical approval is needed for research that involves physical intervention, biological, genetic, biometric, or sensitive personal data. Since LegiLexi data do not contain any sensitive personal information and cannot be connected to any individual student, no ethical approval for this study was needed. This was confirmed by a statement from the Swedish Ethical Review Authority.

Participants

Our initial dataset included 147,215 students in grades 1-3, whose teachers had registered their class for LegiLexi testing sometime from fall 2016 to spring 2021. We excluded students from the academic year 2016-2017 (when the test was still paper-based), those whose schools could not be matched to a socioeconomic index value (see below), and those with a class-ID with five or fewer teacher-registered students to avoid including e.g., practice students, and were left with 126,785 students. Then we excluded whole class-IDs with no tested students at all (i.e., teachers who never started using LegiLexi after the initial registration of students). We were then left with 126,315 teacher-registered students.

To answer our specific research questions, the following students were excluded from the analyses: those who had not participated in the targeted subtests (NA on the subtests decoding and/or reading comprehension), first-grade students who had a score of 0 on the easier reading comprehension test (subtest 5) but a non-zero score for the harder reading comprehension test (subtest 6) since only subtest 5 was analysed for this age group, and students with the unlikely recorded test score of 144 (maximum) at the word decoding test.

The final sample consisted of 97,073 students from 248 different municipalities, 1277 schools and 5250 classrooms. This final sample was compared to the teacher-registered list of students (126,315), and the proportion of included students was comparable between all academic years: 2020-2021 (71.92%), 2019-2020 (73.40%), 2018-2019 (73.03%), 2017-2018 (72.49%).

Socioeconomic index

To get an estimation of students’ socioeconomic background, the socioeconomic index (SEI) for each school and year published by the Swedish Department of Education was used (for data for 2020 see Skolverket, 2021). The statistical model for SEI is created by Statistics Sweden (2020). and includes many aspects of students’ socioeconomic background: caregivers’ highest level of education, the total income of the household, whether a caregiver received social security benefits, whether the student lives with both parents, socioeconomic status of the housing area, and years since the student immigrated to Sweden (if applicable). The model also includes the student's gender and the number of siblings. A school with more students from a disadvantaged socioeconomic background will get a higher SEI value, and a higher allocation of economic support from the Swedish government (Statistics Sweden, 2020). For research question 2, we included the subset of students in schools with a socioeconomic index >130 (Skolverket, 2018), corresponding to the schools with the highest SEI quartile in Sweden in 2020.

Materials

All tests come from the LegiLexi materials and can be found at www.LegiLexi.org. Teachers follow a manual provided by LegiLexi (Fälth et al., 2017).

Word decoding

The test measuring word decoding ability was performed individually with the test leader (a teacher in the participant's school). The test consists of a list of 144 common real words with increasing length and difficulty. The student is asked to read as many words from the list as possible in one minute and the test leader notes the number of correctly read words. Test-retest correlation reported for grades 1-3 was r = 0.87-0.88 (Fälth et al., 2017).

Reading comprehension

Both reading comprehension tests are conducted in a group setting and presented digitally on an iPad or a computer screen. For grade 1, the test consists of 12 short texts. After reading each text silently, the student must choose the best fitting picture out of five similar pictures. The texts increase in length (from one sentence to nine sentences) and difficulty (from LIX 3 to LIX 18 where LIX < 30 is classified as very easy to read such as children's books). The test time is limited to 5 minutes. The maximum score is 12, and the test-retest correlation reported for grade 1 is r = 0.75 (Fälth et al., 2017). For grades 2 and 3, the test consists of four texts, with three multiple-choice questions attached to each text. The test time is limited to 7 minutes. The maximum score is 18, and the test-retest correlation reported for grades 2-3 is r = .77-78 (Fälth et al., 2017). Both tests measure the student's ability to find information (read what is in the line) and make inferences or draw conclusions (read between the lines).

Statistical analysis

Prior analyses, we defined what we would consider a relevant difference in reading skills, and chose to define a negative effect as a difference compared to the pre-pandemic comparison group with 0.2 standard deviations (SD) or more (approximately the lower bound of a small effect) in decoding and/or in reading comprehension results (see e.g., Lakens, Scheel, and Isager (2018), for a discussion on the smallest effect size of interest (SESOI)). Given the large data set, it is likely that some differences will be statistically significant, but this does not necessarily mean that the difference has any practical implications, and thus it is crucial to investigate the size of the difference. We believe a lower boundary of 0.2 SD is a minimum benchmark to test against. The cross-sectional data were analysed with pair-wise Games-Howell tests, and the longitudinal data were analysed with ANCOVAs including the covariates gender, socioeconomic index (SEI) and whether a student follows the curriculum for Swedish as a second language (SSL) or not.

Results

Our first research question was regarding the reading abilities of the whole sample of students in schools with the full range of SEI values, and the first sub-question addressed whether reading scores were lower during the pandemic compared to before the pandemic.

To present an overview of the cross-sectional data (N = 97,073), Table 1 presents means and standard deviations for word decoding and reading comprehension (results from the easier reading comprehension test for grade 1 and the more difficult one for grades 2 and 3, see under materials above) for the pandemic academic year 2020-2021 compared to the average test results from the academic years 2017-2018, 2018-2019 and 2019-2020, and the difference in test scores expressed in Cohen's d. The results did not support a learning loss during the pandemic - results from both decoding and reading comprehension were consistently higher in the academic year 2020-2021 compared to previous years. The effect sizes were not close to the chosen smallest effect size of interest (d = 0.2) however, except for the results from the first test period in first grade, which are addressed in the discussion.

Table 1

Means and standard deviations for word decoding and reading comprehension for the pandemic academic year 2020-2021 and the comparison academic years 2017-2018, 2018-2019 and 2019-2020. The difference in test scores is expressed in Cohen's d. N = 97073.

			Pandemic		Comparison
Test	Grade	Period	M	SD	M	SD	d
Reading comprehension	1	Fall	4.44	3.00	3.81	3.05	0.21
Reading comprehension	1	Winter	6.20	3.22	5.96	3.22	0.07
Reading comprehension	1	Spring	7.61	3.12	7.32	3.18	0.09
Reading comprehension	2	Fall	7.92	3.89	7.65	3.86	0.07
Reading comprehension	2	Winter	9.64	4.02	9.33	4.03	0.08
Reading comprehension	2	Spring	10.84	4.01	10.48	4.00	0.09
Reading comprehension	3	Fall	11.09	3.97	10.80	3.97	0.07
Reading comprehension	3	Winter	12.37	3.77	11.81	3.75	0.15
Reading comprehension	3	Spring	12.92	3.70	12.54	3.62	0.11
Word decoding	1	Fall	27.12	21.28	23.44	20.93	0.18
Word decoding	1	Winter	40.67	22.04	39.75	21.93	0.04
Word decoding	1	Spring	50.66	22.14	49.51	22.61	0.05
Word decoding	2	Fall	55.64	23.44	54.56	23.97	0.04
Word decoding	2	Winter	65.93	23.03	65.65	23.45	0.01
Word decoding	2	Spring	72.54	22.33	71.67	22.86	0.04
Word decoding	3	Fall	75.30	23.12	73.65	23.19	0.07
Word decoding	3	Winter	82.22	22.30	80.55	22.79	0.07
Word decoding	3	Spring	86.48	22.22	85.62	22.87	0.04

Table 2 presents descriptive data as well as test results for decoding and reading comprehension for each grade and test period by academic year.

Table 2

Descriptive data and test results for word decoding and reading comprehension, split by grade, test period, and academic year. N = 97073.

					Proportions			Word decoding		Reading comprehension
Grade	Period	Year	n	SEI	Females	SSL	WD -1 SD	M	SD	M	SD
1	Fall	20/21	8076	95.11	0.50	0.14	0.10	27.12	21.28	4.44	3.00
1	Fall	19/20	5340	95.94	0.49	0.15	0.21	22.38	20.68	3.75	3.14
1	Fall	18/19	3330	95.60	0.48	0.14	0.14	24.39	21.26	3.84	2.96
1	Fall	17/18	894	90.79	0.48	0.18	0.05	26.18	20.80	4.00	2.78
1	Winter	20/21	13148	94.16	0.49	0.13	0.12	40.67	22.04	6.20	3.22
1	Winter	19/20	9064	92.61	0.49	0.13	0.13	39.90	21.82	6.18	3.23
1	Winter	18/19	5660	95.17	0.49	0.14	0.14	39.31	22.15	5.80	3.26
1	Winter	17/18	2089	90.22	0.49	0.13	0.14	40.24	21.76	5.48	2.95
1	Spring	20/21	14658	95.27	0.49	0.14	0.13	50.66	22.14	7.61	3.12
1	Spring	19/20	10255	93.37	0.49	0.14	0.15	50.22	22.55	7.55	3.15
1	Spring	18/19	5477	99.30	0.49	0.16	0.16	48.88	22.46	7.15	3.20
1	Spring	17/18	2183	93.65	0.49	0.12	0.18	47.75	23.12	6.67	3.15
2	Fall	20/21	16745	94.69	0.49	0.15	0.14	55.64	23.44	7.92	3.89
2	Fall	19/20	10584	98.58	0.49	0.16	0.16	54.45	24.28	7.57	3.87
2	Fall	18/19	5241	97.03	0.48	0.16	0.16	54.42	23.49	7.69	3.86
2	Fall	17/18	2690	91.14	0.49	0.16	0.15	55.30	23.64	7.93	3.79
2	Winter	20/21	16264	98.01	0.49	0.15	0.16	65.93	23.03	9.64	4.02
2	Winter	19/20	11351	98.15	0.50	0.15	0.16	65.94	23.58	9.28	3.99
2	Winter	18/19	5779	98.86	0.49	0.16	0.17	64.76	23.34	9.22	4.11
2	Winter	17/18	3563	88.39	0.49	0.15	0.15	66.15	23.20	9.65	4.00
2	Spring	20/21	15927	96.43	0.49	0.14	0.15	72.54	22.33	10.84	4.01
2	Spring	19/20	11489	98.86	0.50	0.15	0.16	72.17	22.78	10.47	3.94
2	Spring	18/19	5129	102.38	0.49	0.17	0.16	71.29	22.84	10.44	4.07
2	Spring	17/18	3818	99.06	0.48	0.19	0.17	70.69	23.09	10.53	4.06
3	Fall	20/21	16631	99.12	0.49	0.16	0.16	75.30	23.12	11.09	3.97
3	Fall	19/20	7279	100.14	0.49	0.16	0.17	73.34	23.09	10.77	3.97
3	Fall	18/19	7037	98.58	0.49	0.16	0.16	74.01	23.33	10.80	4.00
3	Fall	17/18	1913	96.87	0.50	0.16	0.17	73.52	23.04	10.93	3.87
3	Winter	20/21	13383	100.51	0.49	0.16	0.15	82.22	22.30	12.37	3.77
3	Winter	19/20	5037	100.35	0.50	0.16	0.16	81.79	22.49	11.89	3.71
3	Winter	18/19	5603	101.06	0.48	0.18	0.16	80.35	22.77	11.73	3.80
3	Winter	17/18	2005	113.74	0.52	0.22	0.19	78.00	23.34	11.80	3.71
3	Spring	20/21	9180	105.24	0.49	0.17	0.15	86.48	22.22	12.92	3.70
3	Spring	19/20	4170	98.91	0.50	0.16	0.14	86.55	21.84	12.65	3.55
3	Spring	18/19	3198	110.88	0.49	0.22	0.15	86.12	22.36	12.40	3.66
3	Spring	17/18	1606	116.07	0.50	0.24	0.18	82.20	25.95	12.52	3.72

Notes: SEI: Mean socioeconomic index, higher value indicates more students from disadvantaged socioeconomic backgrounds. SSL: Students who follow Swedish as a second language (SSL) curriculum. WD -1 SD: Proportion students with word decoding results -1 SD below the mean or lower.

Comparing the results in Table 2 for each test period and academic year, the results presented in the overview in Table 1 hold: average reading scores (word decoding and reading comprehension) during the four test periods during the pandemic (spring 2019 to spring 2021) were slightly higher compared to previous years. Since the direction of differences between years were not indicating a learning loss during the COVID-19 pandemic, we do not present the results of any statistical tests here, but the visual distribution of test scores for each academic year and test period, as well as the results of significant pairwise Games-Howell tests, are provided in supplementary materials A and B.

Our second sub-question was regarding the proportion of students with weak decoding skills, and whether this proportion increased during the pandemic. To answer this question, we used the average standard deviation for each test period for the academic years 2017-2018, 2018-2019, and 2019-2020 (see Table 1), to calculate the proportion of students who scored one standard deviation below the mean or lower at each test period. These results are also presented in Table 2. Again, the proportion was very similar between years, and in general, the proportion of students with weak decoding skills was somewhat lower during the four pandemic test periods. An exception was (again) the first test period in the first grade during the pandemic, with a smaller proportion of students with weak decoding skills, and this is addressed in the discussion.

Longitudinal data (a subset of students)

To complement the cross-sectional data and to be able to control for covariates (gender, socioeconomic index (SEI), whether a student followed the Swedish as a second language (SSL) curriculum), we selected the subset of students who had test results from all three test periods in each grade. This subset of students included 33973 unique students or 35.0% of the total number of students in the full dataset. Note that some students were included in more than one analysis (e.g., a student included in the first grade subsample of the academic year 2018-2019 was also included in the second grade subsample in the academic year 2019-2020 if they were tested during all six test periods). A presentation of the descriptive data for this subset by grade, test period and academic year can be found in supplementary material C. The relevant characteristics of this subsample were similar to the characteristics in the full cross-sectional sample. In total, six ANCOVAs (two per grade) were conducted to determine whether reading ability (decoding and reading comprehension scores) in the three test periods was lower in the pandemic year 2020-2021 compared to the three previous academic years. Interactions between academic year and test period and the covariates were also included in the models. The main effect of interest was the effect of academic year, and the results for this effect are presented below. In addition, for all six models there were significant main effects of test period (i.e., students’ reading ability improved over time), and significant effects of the covariates (see supplementary material D).

For first-grade students (n = 10,878), the main effect of academic year on word decoding was significant, F(3, 10871) = 19.71, p < .001, with a small effect size (generalised η2 = 0.005), as was the main effect of academic year on reading comprehension, F(3, 10871) = 42.07, p < .001, with a small effect size (generalised η2 = 0.009). Similar to the cross-sectional data, the direction of the difference was that the scores were higher in the academic year 2020-2021 compared to previous years, see Fig. 1, Fig. 2 .

Fig. 1

Longitudinal data showing first grade students’ unadjusted word decoding scores, (n = 10,878).

Fig. 2

Longitudinal data showing first grade students’ unadjusted reading comprehension scores,(n = 10,878).

The results for second-grade students (n = 19,945) were similar. There was a significant main effect of academic year on word decoding, F(3, 19938) = 5.77, p < .001, generalised η2 < 0.001, and on reading comprehension, F(3, 19938) = 10.96, p < .001, generalised η2 = 0.001. Again, the effect sizes were very small, and the direction of the differences were in favour of the academic year 2020-2021, see Fig. 3, Fig. 4 .

Fig. 3

Longitudinal data showing second grade students’ unadjusted word decoding scores, (n = 19,945).

Fig. 4

Longitudinal data showing second grade students’ unadjusted reading comprehension scores,(n = 19,945).

Finally, for third-grade students (n = 11,656), there was also a significant main effect of academic year on word decoding, F(3, 11649) = 3.02, p = .029, with a small effect size (generalised η2 < 0.001). There was also a significant main effect of academic year on reading comprehension, F(3, 11649) = 10.43, p < .001, generalised η2 = 0.002. The mean scores varied slightly more for this age-group, but the effect sizes associated with academic year were even smaller than for first- and second-grade students, see Figs. 5 and 6.

Fig. 5

Longitudinal data showing third grade students’ unadjusted word decoding scores, (n = 11,656).

Fig. 6

Longitudinal data showing third grade students’ unadjusted reading comprehension scores,(n = 11,656).

Students from schools serving a large proportion of low socio-economic status households

Our second research question was regarding the reading abilities of students in schools serving a large proportion of low socio-economic status households addressing the same sub-questions as for the whole sample. Table 3 and Table 4 present means and standard deviations for word decoding and reading comprehension in the subset of students from schools with a socioeconomic index score of 130 or higher (n = 25,330). Note that the proportion of students following the Swedish as a second language (SSL) curriculum was markedly higher than in the whole sample, and overall, the average reading scores were lower than for the whole sample. However, the results do not support a pandemic learning loss in this group of Swedish students either. In grades 1 and 2, the scores for both word decoding and reading comprehension are consistently slightly higher in the test periods during the pandemic, and in grade 3 the scores are very similar across years, but there is a bit more variety in which year has the highest average number, and consequently, there is a minuscule negative difference at the spring test period in grade 3 (d = -0.01 SD), but still far from our chosen smallest effect size of interest. Just as for the whole sample, the proportion of students scoring 1 SD or lower on decoding is not larger during the pandemic (Table 4), but rather tends to be larger during the academic years 2017-2018 and 2018-2019 compared to 2019-2020 and 2020-2021, possibly because fewer students were tested in the earlier years. Because the standard deviation is large, no first-grade students in the fall testing fall at or below -1 SD, but this number stabilizes from the spring testing in year 1 and onwards, a similar pattern as in the data for the whole sample.

Table 3

Means and standard deviations for word decoding and reading comprehension for the pandemic academic year 2020-2021 and the comparison academic years 2017-2018, 2018-2019 and 2019-2020. The difference in test scores is expressed in Cohen's d. Only students with SEI >130, N = 25330.

			Pandemic		Comparison
Test	Grade	Period	M	SD	M	SD	d
Reading comprehension	1	Fall	3.60	2.64	2.98	2.60	0.24
Reading comprehension	1	Winter	5.36	3.12	4.77	3.10	0.19
Reading comprehension	1	Spring	6.62	3.22	6.07	3.26	0.17
Reading comprehension	2	Fall	6.78	3.56	6.53	3.54	0.07
Reading comprehension	2	Winter	8.36	3.83	7.99	3.87	0.10
Reading comprehension	2	Spring	9.49	4.02	9.18	3.98	0.08
Reading comprehension	3	Fall	9.56	4.04	9.47	4.10	0.02
Reading comprehension	3	Winter	10.91	4.04	10.42	3.96	0.12
Reading comprehension	3	Spring	11.42	4.03	11.29	3.94	0.03
Word decoding	1	Fall	21.84	19.56	18.21	18.88	0.19
Word decoding	1	Winter	36.50	22.04	33.49	21.42	0.14
Word decoding	1	Spring	46.91	22.42	43.76	22.81	0.14
Word decoding	2	Fall	50.20	24.20	48.25	24.42	0.08
Word decoding	2	Winter	60.85	24.02	60.29	24.10	0.02
Word decoding	2	Spring	67.95	22.99	66.95	23.66	0.04
Word decoding	3	Fall	69.92	24.47	69.61	24.68	0.01
Word decoding	3	Winter	77.41	23.26	76.67	24.22	0.03
Word decoding	3	Spring	81.15	23.11	81.38	23.46	-0.01

Table 4

Descriptive data and test results for word decoding and reading comprehension, split by grade, test period and academic year. Only students with SEI > 130, n = 25,330.

					Proportions			Word decoding		Reading comprehension
Grade	Period	Year	n	SEI	Females	SSL	WD -1 SD	M	SD	M	SD
1	Fall	20/21	1923	179.77	0.49	0.41	0.00	21.84	19.56	3.60	2.64
1	Fall	19/20	1327	181.47	0.46	0.44	0.00	17.22	18.67	2.94	2.72
1	Fall	18/19	849	180.80	0.49	0.38	0.00	19.85	19.23	3.03	2.54
1	Fall	17/18	191	189.85	0.48	0.63	0.00	17.76	18.29	3.02	1.88
1	Winter	20/21	2918	182.88	0.48	0.39	0.13	36.50	22.04	5.36	3.12
1	Winter	19/20	2081	178.43	0.50	0.39	0.16	33.82	21.46	4.98	3.17
1	Winter	18/19	1276	178.51	0.50	0.37	0.17	32.86	21.18	4.58	3.09
1	Winter	17/18	467	183.10	0.49	0.40	0.17	33.72	21.88	4.32	2.67
1	Spring	20/21	3428	180.64	0.49	0.40	0.11	46.91	22.42	6.62	3.22
1	Spring	19/20	2338	177.03	0.49	0.42	0.14	44.45	22.52	6.34	3.28
1	Spring	18/19	1330	182.29	0.50	0.43	0.18	42.51	22.78	5.78	3.25
1	Spring	17/18	471	185.01	0.48	0.26	0.18	43.80	24.18	5.55	3.07
2	Fall	20/21	3859	181.20	0.48	0.42	0.13	50.20	24.20	6.78	3.56
2	Fall	19/20	2788	180.08	0.49	0.41	0.16	48.60	24.82	6.45	3.62
2	Fall	18/19	1339	182.88	0.47	0.41	0.17	47.89	23.64	6.71	3.46
2	Fall	17/18	522	188.59	0.54	0.54	0.19	47.30	24.26	6.51	3.22
2	Winter	20/21	4054	184.70	0.49	0.43	0.16	60.85	24.02	8.36	3.83
2	Winter	19/20	2967	179.02	0.50	0.40	0.17	60.47	24.35	7.92	3.84
2	Winter	18/19	1477	184.94	0.48	0.43	0.17	59.85	24.05	8.02	4.06
2	Winter	17/18	834	176.07	0.51	0.41	0.15	60.45	23.31	8.22	3.62
2	Spring	20/21	3848	179.87	0.50	0.39	0.14	67.95	22.99	9.49	4.02
2	Spring	19/20	3007	177.03	0.51	0.38	0.15	67.30	23.25	9.20	3.96
2	Spring	18/19	1443	182.22	0.49	0.40	0.17	66.76	23.95	9.23	4.09
2	Spring	17/18	1141	182.10	0.51	0.45	0.17	66.27	24.34	9.05	3.89
3	Fall	20/21	4328	183.09	0.50	0.42	0.16	69.92	24.47	9.56	4.04
3	Fall	19/20	2149	179.50	0.49	0.41	0.18	68.40	24.10	9.45	4.07
3	Fall	18/19	2002	180.44	0.50	0.40	0.14	71.16	25.02	9.46	4.10
3	Fall	17/18	380	200.02	0.52	0.47	0.17	68.30	25.66	9.65	4.23
3	Winter	20/21	3643	183.76	0.49	0.40	0.16	77.41	23.26	10.91	4.04
3	Winter	19/20	1371	174.80	0.48	0.42	0.16	77.64	23.06	10.55	4.01
3	Winter	18/19	1705	181.59	0.48	0.44	0.17	77.47	24.83	10.36	3.90
3	Winter	17/18	674	198.97	0.51	0.47	0.19	72.69	24.58	10.28	3.98
3	Spring	20/21	2491	188.93	0.50	0.44	0.16	81.15	23.11	11.42	4.03
3	Spring	19/20	1150	178.55	0.50	0.40	0.16	82.94	22.96	11.39	3.89
3	Spring	18/19	1015	190.46	0.47	0.47	0.16	80.93	22.62	11.05	3.95
3	Spring	17/18	681	181.97	0.49	0.45	0.17	79.42	25.32	11.47	3.99

Notes: SEI: Mean socioeconomic index, higher value indicates more students from disadvantaged socioeconomic backgrounds. SSL: Students who follow Swedish as a second language (SSL) curriculum. WD -1 SD: Proportion students with word decoding results -1 SD below the mean or lower.

Finally, we chose the subset of students from schools with an SEI >130 who were tested all three times during one academic year. This subset of students included 8382 unique students or 33.1% of the total number of students in the cross-sectional subset with SEI > 130, and we analysed these results by grade, just as we did for the larger sample. There was no evidence of a negative effect on word decoding and reading comprehension of the COVID-19 pandemic including gender, SEI and SSL as covariates in this longitudinal subset either. Descriptive data and ANCOVA tables and figures related to this longitudinal subset can be found in supplementary materials E and F.

Discussion

The COVID-19 pandemic resulted in a clear disruption to “business as usual” in K–12 education in a majority of the world's countries (UNESCO, 2020), but not in Sweden which chose to keep preschools, primary schools, and lower secondary schools open during the pandemic. It is of great political interest to know whether students can have their educational needs met in the circumstances to which the pandemic nevertheless contributed.

Here, using data from the freely available and widely used LegiLexi reading assessment tool (Fälth et al., 2017), we demonstrate no general learning loss when comparing reading skills before and during the pandemic in primary school children in Sweden. The large dataset (N = 97,073) included students from 248 of Sweden's 290 municipalities and is the most representative available dataset on Swedish primary school students’ reading abilities during the COVID-19 pandemic. The data are also unique in that the same skills measured with the same tests (word decoding and reading comprehension) can be followed in large groups of students over time. This is in contrast to several other studies investigating or modelling the effect of the pandemic, which are sometimes relying on modelling data from conceptually different tests of achievements, and/or to input missing data (e.g., Engzell et al., 2021; Tomasik et al., 2021). In fact, one study from Australia, where the same reading assessments were used and students and schools were carefully matched, did not find a negative effect of an 8-10 week “learning from home” period on reading comprehension (Gore et al., 2021). The sample was considerably smaller (N = 3030) than in the present study, however. In contrast, Maldonado and De Witte (2021) found a negative effect of d = 0.19 in Dutch test scores, comparing the same assessments in matched samples of Flemish 6th graders at the school level controlling for grade 4 reading scores (N = 1478 schools). Thus, it is likely that the estimates are confounded since most studies are observational; many other factors apart from school closures and/or the effectiveness of online learning may affect the observed outcomes. The present study is also observational, with the associated limitations and lack of control over e.g., reading instruction given and inclusion and exclusion of students at any given time point. But given the large and representative dataset, and the uniformity of results across several academic years and multiple testing periods across the chosen analyses, we are reasonably confident in our conclusion that COVID-19 did not cause a general learning loss regarding reading ability among Swedish primary grade students.

In addition, the results of the present study did not indicate a learning loss for the subset of students from more disadvantaged backgrounds either, which both UNESCO and previous research show are especially at risk for learning disruptions in exceptional circumstances, such as school closures during a pandemic (Engzell et al., 2021; Maldonado & De Witte, 2021). Since the beginning of the COVID-19 pandemic, students across the globe were affected by sudden and radical changes in all areas of their life, but the big difference for Swedish students was that the schools did not close, thus providing a stable foundation for everyday life during the pandemic, also for the more vulnerable groups of students.

The fact that results from both word decoding and reading comprehension tests were consistently higher in the pandemic academic year 2020-2021 compared to previous academic years is interesting and warrants discussion. This was found both in the full sample and in the longitudinal sample, even though the effect sizes were in general very small. Accordingly, we concluded that this finding lacks practical significance (Lakens et al., 2018). There was one exception, however: the fall testing in year 1 showed threshold or near threshold effect sizes for both word decoding (d = 0.18) and reading comprehension (d = 0.21), in favour of the pandemic academic year 2020-2021. Further analyses showed that the proportion of included students (who had scores for both subtests) for this test period (53%) was lower compared to the academic years before the pandemic (63%), a difference not seen in the other test periods in grade 1 and markedly lower than the average proportion of tested students out of teacher-registered students which were above 70% across all included academic years. It is, therefore, possible that teachers chose to only use the National Agency for Education's compulsory assessment material for reading at the beginning of fall 2020 to a larger extent, and/or chose not to test students who have not yet “cracked the code” with the word decoding and reading comprehension sub-tests in LegiLexi. In other words, it is likely that fewer students with weaker reading skills are included in the fall grade 1 sample for the pandemic academic year 2020-2021, which is further confirmed by the smaller proportion of students with -1 SD or lower scores on decoding. It is worth noting that the grade 1 fall testing in 2017 shows a similar pattern with both a lower inclusion rate and a smaller proportion of weak readers, which indicates that this was not an effect specific to the pandemic. It is important to note here, that this lower inclusion rate was only evident in the first test period in grade one, and not the subsequent two test periods. In addition, since the second wave of COVID-19 in Sweden did not take off until the end of October 2020 (with a culmination in late December 2020 according to data from the Public Health Agency of Sweden (Folkhälsomyndigheten, 2021)), the lower inclusion rate for the fall testing is not likely to be a direct consequence of the pandemic, e.g., due to sick and absent students. The subsequent winter and spring test periods in grade one during the pandemic academic year do not show any differences in proportion of included students compared to previous years either. Decoding and reading comprehension scores for grade 2 and 3 are also very similar across the different academic years, which indicate that the general results are representative and comparable. But importantly, we conclude that the larger effect size in the grade 1 fall testing in 2020 does not represent improved reading skills during the pandemic.

Across all included academic years, the proportion of included students out of teacher-registered students remained stable at just above 70%, although the pandemic academic year 2020/2021 had a slightly lower inclusion rate. In 2020/2021 the number of teachers who registered to use the LegiLexi testing tool increased dramatically as well, so even though the proportion of included students was somewhat lower, the total number of students was almost as high in 2020/2021 as all previous years combined. It is possible, however, that some of the most disadvantaged and at-risk students, who endured the most severe consequences of the pandemic (e.g., with sickness and/or death in the family as well as own sickness and therefore much absence in school) are not represented in the data material, which might overestimate the reading skills of students during the pandemic. Since the difference in included students for the pandemic year compared to previous years is small (≈0.5-1.5%) it is not likely, however, that the inclusion of those students would dramatically change the general conclusions of the present study.

Conclusion

Mitigating a pandemic in a global society has proven to be very challenging, and the different COVID-19 strategies will most likely be evaluated and discussed for years to come. We show that primary school students’ reading skills were not negatively affected by the pandemic when schools were kept open in Sweden, also in the more vulnerable groups who were disproportionally affected by the pandemic in other aspects of their lives (Sjögren et al., 2021). Importantly, data from the first wave of the pandemic showed that the incidence of severe COVID-19 in preschool children and primary school students in Sweden was very low, even though schools were kept open (Ludvigsson et al., 2021).

We conclude that there is no evidence of a learning loss regarding early reading skills in Swedish primary school students. We are of course not claiming that the COVID-19 pandemic did not have any negative effect on reading ability of any individual Swedish primary school student. But given that the overall student inclusion rates are so similar between years, that the proportion of students with word decoding scores of 1 SD below the mean or lower and the general distribution of test scores are stable across test periods and years, we conclude that Swedish primary school students’ reading skills stayed at a stable level during the pandemic. The same conclusion applies to students from more disadvantaged backgrounds. In the light of international studies on reading skills in younger students during the pandemic, we conclude that the decision to keep schools open benefitted Swedish primary school students. This decision might also have mitigated other potentially negative effects of school closures, especially for students from more disadvantaged backgrounds (UNESCO, 2021a).

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.