The impact of school closure intervention during the third wave of the COVID-19 pandemic in Italy: Evidence from the Milan area.

BACKGROUND: In February 2021, the spread of a new variant of SARS-CoV-2 in the Lombardy Region, Italy caused concerns about school-aged children as a source of contagion, leading local authorities to adopt an extraordinary school closure measure. This generated a debate about the usefulness of such an intervention in light of the trade-off between its related benefits and costs (e.g. delays in educational attainment, impact on children and families' psycho-physical well-being). This article analyses the epidemiological impact of the school closure intervention in the Milan metropolitan area.
METHODS: Data from the Agency for Health Protection of the Metropolitan City of Milan allowed analysing the trend of contagion in different age classes before and after the intervention, adopting an interrupted times series design, providing a quasi-experimental counterfactual scenario. Segmented Poisson regression models of daily incident cases were performed separately for the 3-11-year-old, the 12-19-year-old, and the 20+-year-old age groups, examining the change in the contagion curves after the intervention, adjusting for time-varying confounders. Kaplan-Meier survival curves and Cox regression were used to assess the equality of survival curves in the three age groups before and after the intervention.
RESULTS: Net of time-varying confounders, the intervention produced a daily reduction of the risk of contagion by 4% in those aged 3-11 and 12-19 (IRR = 0·96) and by 3% in those aged 20 or more (IRR = 0·97). More importantly, there were differences in the temporal order of contagion decrease between the age groups, with the epidemic curve lowering first in the school-aged children directly affected by the intervention, and only subsequently in the adult population, which presumably indirectly benefitted from the reduction of contagion among children.
CONCLUSION: Though it was not possible to completely discern the effect of school closures from concurrent policy measures, a substantial decrease in the contagion curves was clearly detected after the intervention. The extent to which the slowdown of infections counterbalanced the social costs of the policy remains unclear.

Introduction

In February 2021, a warning arose in the Lombardy Region, Italy, when four outbreaks of a new variant of SARS-CoV-2 (lineage B·1·1·7) were identified in some schools in the municipalities of Bollate (province of Milan), Castrezzato (Brescia), Mede (Pavia), and Viggiù (Varese). Recent literature warned that the so-called Alpha variant was more contagious than previous ones, with an easier spread among children [1, 2]. In the four municipalities, the increasing number of infected people led to exceeding the alarm threshold established by the Ministry of Health of 250 cases per 100,000 inhabitants, with the consequent implementation of restrictions on mobility and closures of public services (lockdown provisions) from February 17. Together with these restrictions, local authorities ordered the closure of schools of each level of education in the municipalities where the outbreaks took place, declaring them a ‘red zone’–previously, kindergartens (3–5 years old), primary schools (6–10 years old), and the first level of secondary school (11 years old) have always remained open in Lombardy, starting from September 2020. In the following days, a widespread increase in the number of infected people was observed in the whole region, leading local authorities to impose a total school closure in Lombardy from March 5, which lasted until April 5.

School closure—with educational activities carried out through distance learning—implies important social costs; thus, it is crucial to understand the efficacy of such interventions considering the trade-off between the flattening of the epidemic curve and the longstanding consequences on children’s education and well-being. Missing out on formal learning could have an impact on children’s mental health, likely reducing self-control, social competence, and logical deduction amongst other cognitive abilities [3-5] as well as increasing educational inequalities emphasising different resilience between socioeconomically advantaged and disadvantaged families and contexts [6].

Analysing different countries, several scholars have stressed the role of school closures in the mitigation of the COVID-19 spread [7-9], referring to the spreading role of asymptomatic children and the relational school contexts as a nidus of transmission [10], while others estimated a light effect on the spread of COVID-19, especially if compared to the high social cost of this policy [4, 11–13]. However, most of the studies that have been carried out suffer from several limitations—acknowledged by the authors themselves—which make it difficult to understand the real impact of school closure interventions on the spread of contagion. From a methodological standpoint, disentangling the impact of school closures from the impact of other interventions carried out simultaneously might not be feasible, as it is seldom possible to isolate the effect of one containment measure from the others. Moreover, some variants resulted in being associated with higher spread in specific age classes, which are generally less exposed to contagion, so that aggregated data might conceal specific trends. Indeed, some scholars raised the alarm about the high contagiousness of the new variant identified first in the United Kingdom in late summer 2020, as ‘ [i]n September, 2020, this variant represented just one in four new diagnoses of COVID-19, whereas by mid-December, this had increased to almost two thirds of new cases in London’ [1].

The analysis of school closure interventions lacks counter factuality, and the impact of this kind of policy continues to be unproved or debated, mainly because of the presence of time-variant confounders that could explain the decrease in contagion after the school closures. In this sense, without an experimental design, it is impossible to test the independent role of school closures in shaping the contagion curve. However, some situations permit evaluating a quasi-experimental model considering two homogeneous contextual conditions which are subject to different health policy interventions.

Since November 6, 2020, the Italian government has adopted a novel system of management for pandemic-related interventions according to four different scenarios (‘white’, ‘yellow’, ‘orange’, and ‘red’ zones), with gradual regional restrictions corresponding to biweekly R values (< 0·5, 0·5–1, 1–1·25, > 1·5), as summarised in Table 1. As regards Lombardy, the school closure intervention on 5 March was imposed in the context of a shift from ‘orange’ to ‘strengthen orange’ zone (conceived ad hoc), with background restriction measures left almost untouched. All levels of education were moved to distance learning and only other minor restrictions were introduced, namely the prohibition against (i) using playgrounds in public parks, though recent literature has shown that the risk of outdoor contagion appears to be very low compared to indoor settings [14, 15]; (ii) reaching second homes, which may reasonably have had a limited impact; (iii) visiting friends and relatives without a ‘proven’ necessity, which needed only to be self-certified with little possibility of control. Bearing in mind that the ‘orange’ zone before the shift towards the ‘strengthen orange’ zone lasted only four days, it is nonetheless important to consider the restrictions previously applied, falling under the ‘yellow’ zone scenario, and how they differed from the ones in the subsequent scenarios. Three changes apply in the shift from a ‘yellow’ zone to an ‘orange’ zone scenario, namely: (i) the closure of museums, which were previously already closed on weekends; (ii) the closure of bars and restaurants, which were previously already closed after 6 p.m.; (iii) the prohibition against moving outside the municipality of residence. This last measure, which appears to be the most stringent one, was actually subject to several exceptions due to reasons that needed only to be self-certified (work motivations, health issues and unspecified urgent reasons could be declared) and did not apply for people living in municipalities with less than 5,000 inhabitants, who could move within a radius of 30 km from their residence. Moreover, for a remarkable portion of the population living in the municipality of Milan (~1.4 million inhabitants; ~182 km2; ~40% of the study area population), this did not alter daily mobility habits.

Table 1

Summary of the main restrictive measures applied in each scenario, with relative dates of occurrences in the Lombardy region (changes with respect to the previous scenario in bold).

‘White’ zone	‘Yellow’ zone	‘Orange’ zone	‘Strengthen Orange’ zone	‘Red’ zone
	(1–28 February)	(1–4 March)	(5–14 March)	(15 March–5 April)
Rt < 0·5	0·5 < Rt < 1	1 < Rt < 1·25		Rt > 1·5
• Suspension of activities at higher risk	• Curfew (10 p.m.– 5 a.m.)	• Curfew (10 p.m.–5 a.m.)	• Curfew (10 p.m.– 5 a.m.)	• Curfew (10 p.m.– 5 a.m.)
	• Closure of bars and restaurants at 6 p.m.	• Prohibition against moving outside the municipality of residence (except for ‘proven’ needs)	• Prohibition to move outside the municipality of residence	• Prohibition of unauthorised mobility (also within the municipality of residence)
	• Closure of shopping malls on the weekend	• Closure of bars and restaurants (delivery and take-away only)	• Closure of bars and restaurants (delivery and take-away only)	• Closure of shops and retail markets
	• Closure of movie theatres, theatres	• Closure of movie theatres, theatres, and museums	• Closure of shopping malls on the weekend	• Closure of hairdressers and beauticians
	• Closure of sports centres	• Closure of shopping malls on the weekend	• Closure of movie theatres, theatres, and museums	• Closure of bars and restaurants (delivery and take-away only)
	• Closure of betting places	• Closure of sports centres	• Closure of sports centres	• Closure of shopping malls on the weekend
• Closure of museums on the weekend	• Prohibition on conferences, conventions, and fairs	• Closure of betting places	• Closure of betting places	• Closure of movie theatres, theatres, and museums
• Local quarantines if needed	• Closure of museums on the weekend	• Prohibition on conferences, conventions, and fairs	• Prohibition on conferences, conventions, and fairs	• Closure of sports centres
	• Distance learning for students from the second level of secondary school onward	• Distance learning for students from the second level of secondary school onward	• Prohibition against using recreational areas in public parks	• Closure of betting places
			• Prohibition against reaching second homes	• Prohibition on conferences, conventions, and fairs
			• Prohibition against visiting friends and relatives without a ‘proven’ necessity	• Distance learning for students at all school levels
			• Distance learning for students at all school levels

Note: The effective reproduction number (Rt) represents the average number of new infections caused by a single infected individual at time t in the partially susceptible population. Rt values below 1 indicate that the epidemic is slowing down (each patient infects, on average, less than 1 person), whereas values above 1 indicate that the epidemic is progressing (each patient infects, on average, more than 1 person).

Accordingly, it is clear that other changes are trivial compared to the school closure intervention, which was the only stringent policy applied to a substantial population group (those aged 3–11) with no exceptions. This contributed to shape a ‘natural experiment’ situation, with dates occurring after the intervention conceivable as having been ‘exposed to treatment’ and dates before as ‘controls’. Indeed, as the specific intervention addressed was almost exclusively oriented towards school closure, this makes it possible to estimate the net effect of the school closure policy on the spread of contagion with little distortion—a situation that had never occurred previously in the context of the current pandemic in Italy.

In light of the framework outlined, an interrupted time series (ITS) study design allowed assessing the possible change in the slope of the contagion curve after school closures with unprecedented accuracy. Indeed—though not eliminating all the possible confounding factors—the design adopted permitted us to estimate the impact of the intervention in the correspondence of minor concurrent changes. Individual data from the Agency for Health Protection of the Metropolitan City of Milan (ATS of Milan) were aggregated daily to examine the contagion trend in the population directly affected by the school closures (3–11 years old) and in the remaining population who had completed secondary education (20+ years old), relying on Poisson regression models.

Data and methods

Study population

We analysed data stemming from the ATS of Milan, covering 193 municipalities belonging to the provinces of Milan and Lodi, in the Lombardy region, with an overall population of nearly 3.48 million people in 2020. The study area includes one of the municipalities at the origin of the Alpha variant outbreak, namely Bollate, which has been subject to specific school closures from February 11 and was declared a ‘red’ zone on February 17. The study was performed on the whole ATS population excluding Bollate, whose early Alpha variant outbreak and policy interventions may have influenced the wider area’s results.

Measures

From the Integrated Datawarehouse for COVID Analysis in Milan, we extracted all subjects with nasopharyngeal swab-confirmed (both molecular and antigenic) SARS-CoV-2 infection from 1 February (‘yellow’ zone entry date) as of 5 April (last day of the school closures). Data were aggregated daily to examine trends in incident COVID-19 cases. From the same data source, the overall daily number of tested subjects (both positive and negative) was extracted. The daily number of incident cases and the positivity rate (the share of tests returning a positive result) in the country were extracted from the Italian Civil Protection’s open archive (https://github.com/pcm-dpc/COVID-19). A variable representing the underlying scenario in each date was built relying on the indications available in the national and regional measures to deal with the emergency linked to the spread of the COVID-19 pandemic (https://anci.lombardia.it/dettaglio-news/20202281155-coronavirus-provvedimenti-delle-istituzioni/anci.lombardia.it).

Statistical analysis

To evaluate the impact of school closures on the spread of contagion, we relied on a segmented regression approach in the context of an ITS study design, which is known to be a valuable tool for assessing the effectiveness of population-level health interventions [16-18]. A randomised controlled trial, which among epidemiological study designs is considered the gold standard for evaluating the effectiveness of an intervention, is not feasible in the context of a public health policy targeted at the population level where a random control group has not been previously identified, and cannot be identified ex-post. However, the ITS study design ‘is particularly suited to interventions introduced at a population level over a clearly defined time period and that target population-level health outcomes’ [17], and it is often used in the evaluation of ‘natural experiments’ occurring in real-world settings, where a clear differentiation between the pre- and post-intervention period is present. For our analysis, we followed the methodology proposed by Lopez Bernal and colleagues [16-18]. In dealing with the daily incidence of new diagnosed COVID-19 cases, Poisson regression was used. The basic regression equation is defined as follows: where Y is the outcome (COVID-19 daily incident cases) at time t, T is the number of days elapsed since the beginning of the study, X is a dummy variable indicating the pre-intervention period (coded 0) or the post-intervention period (coded 1), β0 represents the baseline level at T = 0, β1 is interpreted as the change in the outcome associated with a time unit increase (representing the underlying pre-intervention trend), β2 is the level change following the intervention, and β3 indicates the slope change following the intervention (the interaction term between time and intervention: TX). Hypothesising that the intervention would induce a change in the gradient of trend, yet not an immediate change in the level of the outcome, we omitted the dummy variable β2X from our models. Moreover, it is reasonable to assume that the intervention would not exert its effects immediately at the cut-off; hence, we opted for a lagged impact model with slope change only [17]. Bearing in mind the average incubation period of the virus [19-21], we tested each model with a cut-off established at six days from the intervention date.

We tested three main models with the same specification, while differing in age of the population included in the analysis. Firstly, we examined the effect of the intervention in the overall ATS territory without Bollate (192 municipalities) exclusively in children aged between 3–11 years old, that is the population directly affected by the school closures (nursery schools for children younger than 3 years old were not affected by the intervention, whilst higher grades were already put in blended distance learning before the intervention). Secondly, the model was tested in the same area in individuals aged between 12–19 years old, which were subject to a discontinuous alternation between proximate and distance learning before the intervention, according to local epidemic trends. Thirdly, the model was tested in individuals aged 20 years or more to examine the indirect effects of school closures on the spread of contagion in the overall population who had already finished their secondary education. As the daily number of incident cases is influenced by the number of swab tests performed, which is in turn influenced by the day of the week (a lower amount of tests is systematically performed on the weekend, especially on Sunday), the day of the week (to account for seasonality) and the daily number of swab tests (to account for time-varying confounding factors) were fitted as covariates. The models were also adjusted for the scenario of reference (‘yellow’, ‘orange’, ‘strengthen orange’, ‘red’) to take into account background restrictions in observations away from the cut-off. To ensure that the effects detected were independent of the overall local epidemic trend, the models were also adjusted for the daily number of incident cases and the positivity rate in the whole country. The reference population (log transformed) was included in the Poisson models as an offset variable in order to transform the count data into rates. The models were also tested without covariates but with Fourier terms to adjust for seasonality [22], and with a cut-off of 14 days from the intervention, corresponding to the maximum incubation period in the normal range [23]. Overdispersion was allowed by adding a scale parameter, thereby permitting the variance to be proportional rather than equal to the mean. Autocorrelation was checked by plotting the residuals against time and by examining the autocorrelation and partial autocorrelation functions. The model parameters were tested by z-tests and the statistical significance was set by P-values < 0·05 (2-sided); the results were expressed in terms of incidence rate ratios (IRRs). Additionally, we computed Kaplan-Meier survival curves among tested negative subjects to compare the contagion dynamics between the three age groups during the same time interval, also performing Cox regression and Wilcoxon tests to assess the equality of survival curves in the three age groups before and after the intervention. All the analyses were conducted using Stata version 17.

Ethics statement

Ethical approval and consent to participate were not required, as this is an observational study based on data routinely collected by the ATS of Milan, a public body of the Regional Health Service–Lombardy Region. The ATS has among its institutional functions, established by the Lombardy Region legislation (R.L. 23/2015), the government of the care pathway at the individual level in the regional social and health care system, the evaluation of the services provided to, and the outcomes of, patients residing in the covered area. This study is also ethically compliant with the National Law (D.Lgs. 101/2018) and the “General Authorisation to Process Personal Data for Scientific Research Purposes” (nos. 8 and 9 of 2016, referred to in the Data Protection Authority action of December 13, 2018). Data were anonymized with a unique identifier in the different datasets before being used for the analyses.

Results

Descriptive statistics concerning the distribution of cases, swab tests, and positivity rate in the two settings are reported in S1 Table in the S1 File. In the time frame considered (64 days, of which 32 pre- and 32 post-intervention), new 57,331 COVID-19 cases were detected out of 776,534 performed, leading to a 7.4% positive rate.

Fig 1 reports the raw distribution of daily COVID-19 cases before and after the school closure intervention in the three age groups. Together with the overall trend, the scatter plots highlight a noticeable degree of weekly seasonality, as the dots at the bottom of the graphs are related to the systematic reduced number of swab tests performed on Sundays. In all graphs, after a steady increase in the contagion curve, a decline after the intervention is visible. In the 3–11 and 12–19 age class, the decrease is located immediately after the cut-off, whereas in those aged 20 or above a more pronounced lag is present, which is consistent with the fact that the intervention exerted its effects in the first instance on those at whom it was directed (school-aged children), with a subsequent indirect effect on the remaining population.

Fig 1

Scatter plot and fractional polynomial line (degree = 2) with 95% confidence intervals of daily incident COVID-19 cases in the ATS territory (without Bollate) in individuals aged 3–11, 12–19, and 20+ years old or more.

Table 2 reports the results of the main models. The coefficient of interest is the interaction term (TX) between the dummy variable (X) expressing the cut-off and the continuous variable (T) indicating the time elapsed since the start of the study. This represents the change in the slope following the intervention. Specifically, the change in the slope is statistically significant in the models for all the three age groups. On average, the intervention produced a daily reduction of the risk of contagion by 4% in those aged 3–11 and 12–19 (IRR = 0·96) and by 3% in those aged 20 or more (IRR = 0.97). Results are adjusted for seasonality and time-varying confounders, meaning that the effects detected are not attributable to concurrent factors such as the underlying scenario applied, the epidemic trend in the whole country, or the number of swab tests performed locally or nationally. However, these controls cannot exclude the role of other unknown or unconsidered factors that could have influenced the trends (not available in data).

Table 2

Segmented Poisson regression of daily incident COVID-19 cases before and after the school closure intervention (cut-off = six days).

ATS (without Bollate), individuals aged 3–11, 12–19, and 20+ years old or more.

	3–11 y.o.			12–19 y.o.			20+ y.o.
	IRR	P-value	[95% C.I.]	IRR	P-value	[95% C.I.]	IRR	P-value	[95% C.I.]
Cut-off*time	0·96	0·004	[0·94–0·99]	0·96	0·005	[0·94–0·99]	0·97	0·000	[0·96–0·98]
Time	1·02	0·000	[1·01–1·04]	1·03	0·000	[1·02–1·05]	1·02	0·000	[1·01–1·03]
Scenario
’yellow’ (ref·)	1	-	1	1	–	1	1	–	1
’orange’	0·96	0·782	[0·74–1·26]	0·95	0·720	[0·74–1·23]	0·91	0·134	[0·80–1·03]
’strengthen orange’	0·71	0·042	[0·51–0·99]	0·68	0·014	[0·50–0·93]	0·82	0·005	[0·71–0·94]
‘’red’	0·65	0·033	[0·44–0·97]	0·71	0·060	[0·49–1·01]	0·83	0·028	[0·70–0·98]
Weekday
Sunday (ref.)	1	-	1	1	–	1	1	–	1
Monday	10·62	0·000	[5·98–18·87]	6·52	0·000	[3·70–11·47]	1·49	0·019	[1·07–2·08]
Tuesday	10·09	0·000	[6·01–16·93]	5·60	0·000	[3·35–9·38]	2·16	0·000	[1·66–2·80]
Wednesday	8·64	0·000	[5·38–13·88]	5·44	0·000	[3·36–8·80]	2·06	0·000	[1·62–2·62]
Thursday	8·21	0·000	[5·00–13·48]	5·10	0·000	[3·05–8·54]	1·88	0·000	[1·48–2·40]
Friday	8·97	0·000	[5·43–14·82]	5·42	0·000	[3·22–9·11]	1·84	0·000	[1·43–2·38]
Saturday	6·25	0·000	[3·91–9·99]	4·06	0·000	[2·56–6·45]	1·89	0·000	[1·60–2·23]
Swab tests (ATS)	1·00	0·367	[1·00–1·00]	1·00	0·002	[1·00–1·00]	1·00	0·000	[1·00–1·00]
New cases (Italy)	1·00	0·309	[1·00–1·00]	1·00	0·613	[1·00–1·00]	1·00	0·356	[1·00–1·00]
Positivity rate (Italy)	0·94	0·278	[0·85–1·05]	0·94	0·216	[0·85–1·04]	1·04	0·114	[0·99–1·10]

Fig 2 displays the results graphically, showing the predicted values based on the Poisson segmented regression models. The dashed lines report the predictions from the adjusted models presented in Table 2, from which it could be difficult to grasp the change in the trend due to the adjustment for seasonality. Hence, the predictions from unadjusted models have been overlaid on the plot (continuous lines) to facilitate comprehension. As is visible, for all the age groups there is a linear growth in the daily incidence up to the cut-off point set at six days from the intervention, in relation to which a reversal of the pattern is present, with the incidence starting to decrease linearly. Diagnostics, seasonality-adjusted models with Fourier terms, and models with a cut-off set at 14 days from the interventions, are presented in the Supplementary Material (S1–S15 Figs; S2 Table in S1 File).

Fig 2

Interrupted time series of daily incident COVID-19 cases in the ATS territory (without Bollate) in individuals aged 3–11, 12–19, and 20+ years old or more.

Solid line: predicted trend based on the covariate-adjusted segmented Poisson regression model. Dashed line: unadjusted trend.

Finally, the epidemiological curves in the three age groups have been compared using Kaplan-Meier survival functions. The analysis does not include censored cases (subjects who tested negative and/or subjects not tested), so it cannot be used to evaluate the risk of contagion. Rather, it allows for examining the differences in the contagion dynamics between the three age groups during the same time interval, which were found to be statistically significant (Fig 3). These, together with the Cox regression-based test for equality of survival curves [24] (Table 3), provided evidence that the contagion in the 3–11 age group was spreading more rapidly than in the other two age groups before the school closures—with a relative hazard of 1·13, compared to 1·05 for the 12–19 and 0·99 for the 20+ age group—but it became similar to other groups after the intervention, without statistically significant differences. Wilcoxon tests (P-value < 0.001) also confirmed the statistical significance of the differences between the Kaplan-Meier curves.

Fig 3

Kaplan-Meier survival curves of daily incident COVID-19 cases in the three age groups.

Table 3

Cox regression-based test for equality of survival curves in the three age groups before and after the intervention (cut-off = six days).

	Pre-intervention			Post-intervention
	Likelihood Ratio = 36·05			Likelihood Ratio = 0·010
	P-value = 0·000			P-value = 0·996
	Events observed	Events expected	Relative hazard	Events observed	Events expected	Relative hazard
3–11 y.o.	1,741	1,542	1·13	1,118	1,118	1·00
12–19 y.o.	2,611	2,478	1·05	2,122	2,118	1·00
20+ y.o.	24,724	25,056	0·99	25,015	25,019	1·00
Total	29,076	29,076	1·00	28,255	28,255	1·00

Discussion

After the school closures, a considerable decline in daily COVID-19 incidence was visible. This was rather immediate for those directly addressed by the intervention (children 3–11 y.o.), while it was slightly delayed for the adult population (20+ y.o.), with a barely slower decrease, and subjects aged between 12 and 19 years in an intermediate situation. Being aware that the effects detected are not entirely ascribable to the intervention examined due to a portion of unadjustable confounders, it is nonetheless clear that school closure contributed substantially to mitigate the spread of the Alpha variant among the population. This evidence is supported by the following: (i) the presence of a visible discontinuity in the contagion curve right after the intervention, first occurring in school-aged children and only subsequently in the wider population, with the intermediate age group (12–19 y.o.) who experienced an alternation of proximate and distance learning being in an intermediate situation. In relation to this, further evidence was provided by estimating survival curves from individual data, which compared the contagion curves between the three age groups. Specifically, the analysis of relative hazards shows that the raw contagion was spreading more rapidly in the younger group than in other age groups before the intervention and started to slow down earlier after the school closures; (ii) the triviality of concurrent restrictive measures (the prohibition against using playgrounds in public parks, reaching second homes, visiting friends and relatives without a ‘proven’ necessity; the closure of museums, bars, and restaurants; the prohibition against moving outside the municipality of residence), which—as discussed above—introduced minor changes and/or were subject to exceptions, resulting in having concurrent restrictive measures which were less stringent than it might appear.

More than the magnitude of the IRR coefficients emerged from the main models, which were not significantly different between the age groups, it is the temporal order of the reduction in infections between them to suggest a relevant contribution of the school closure intervention in lowering the epidemic curve. Indeed, whilst children have been generally underrepresented among COVID-19 cases in the first year of the pandemic and schools were not identified as a relevant nidus of transmission, the school closure intervention in correspondence of the novel variant highly contagious also among kids led to a lowering of the epidemic curve first in school-aged children directly affected by the intervention, and only subsequently in the adult population, which presumably indirectly benefitted from the reduction in infections among children. The temporal order of the events should also exclude the possibility of other causation mechanisms, such as a decrease in contagion as a consequence of parents forced to stay at home to take care to their children, as otherwise the variations in the age-stratified epidemic curves should be concomitant. Although new variants with greater transmissibility did not seem to give rise to more severe outcomes in children [25, 26], it is nonetheless important to highlight that children could have acted as spreader among the oldest and most fragile population, especially in a context where the national vaccination campaign was at its beginning. On the other hand, given the high social costs imposed by such an intervention in terms of parents’ caregiving burden, children’s education, psychosocial development and mental well-being [27, 28], it is clear that school closure should be imposed only basing on proven evidence of increasing and uncontrollable spread of infection among the school-aged population.

As limitations, we warn that (i) information on SARS-CoV-2-positive individuals was based on the swab test date—and not on the exact onset of the disease—implying a minimum amount of delay intrinsic to our data; (ii) the choice of the cut-off is necessarily somehow arbitrary, as we tested two different thresholds relying on the average and maximum incubation period reported by the literature, but in the lack of information concerning the specific case investigated this may be a source of bias; (iii) though concurrent restrictive measures presumably played a minor role compared to school closures, it is nonetheless important to stress that the IRR detected are likely to overestimate the real impact of the intervention on the decrease of infections, given that it was not possible to take into account the effects of unknown or unmeasured concurrent factors influencing the local epidemic trend.

School closures played a crucial role in reducing SARS-CoV-2 infections during the outbreak of the Alpha variant in the Milan metropolitan area, consequently limiting hospitalisations and casualties. Nonetheless, our study faced the issue from a purely epidemiological standpoint, without addressing the fundamental dispute concerning whether the above-mentioned related social costs exceeded the benefits of intervention, which goes beyond the scope of this study. Despite this, the claim that ‘schools are safe from COVID-19’, which was widely spread among the Italian media, appeared not to be supported by the empirical evidence.

Supporting information–contains all the supporting tables and figures.

(DOCX)

Click here for additional data file.

Minimal dataset.

(CSV)

Click here for additional data file.

20 Apr 2022

The impact of school closure during the third wave of the COVID-19 pandemic in Italy: evidence from the Milan area

PLOS ONE

Dear Dr. Russo,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 04 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Hao Xue

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Please include your tables as part of your main manuscript and remove the individual files. Please note that supplementary tables (should remain/ be uploaded) as separate "supporting information" files.

3. Please ensure that you include a title page within your main document. We do appreciate that you have a title page document uploaded as a separate file, however, as per our author guidelines (http://journals.plos.org/plosone/s/submission-guidelines#loc-title-page) we do require this to be part of the manuscript file itself and not uploaded separately.

Could you therefore please include the title page into the beginning of your manuscript file itself, listing all authors and affiliations.’

4. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

5. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtx.

6. Please amend your authorship list in your manuscript file to include author’s list.

7. Please include your full ethics statement in the ‘Methods’ section of your manuscript file. In your statement, please include the full name of the IRB or ethics committee who approved or waived your study, as well as whether or not you obtained informed written or verbal consent. If consent was waived for your study, please include this information in your statement as well.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: I Don't Know

Reviewer #3: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I would like to congratulate the authors for the study as it would help to take action by the concerned authorities in preventing the spread of the COVID-19 and in pandemics in future too. Though there could be many confounding factors and bias in the decreasing hospitalization and casualties that the authors had mentioned in the limitation. There are also other school of thoughts on the risk vs benefit of social costs and other concerns of school closure.

Overall the study is appreciable in terms of analysis, interpretation and publucatiton.

Reviewer #2: Report for “The impact of school closure during the third wave of the COVID-19 pandemic in Italy: evidence from the Milan area.”

This study evaluates the impact of school closure policy on reducing the spread of COVID-19 pandemic. The authors use an interrupted time series design by comparing right before and right after the implementation of the policy.

Major comments:

1. As acknowledged by the authors, it is very hard to control for other concurrent policies or events in this design. Is it possible to find a region that is close to the Milan area but didn’t implement the policy on March 5th to serve as a control group? We can show that the infection in the control region didn’t change much after March 5th. By doing so, we can at least control for factors such as national level policy variation, change in climate and people’s awareness.

2. Another way to clarify the mechanism is to investigate whether the effect of the policy is more significant in regions with more schools. If the effect is really driven by school closure, I think this should be the case.

3. School closure may force some parents to stay at home and take care of their children, which will also reducing transmission. This can also become a mechanism through which school closure affect transmission.

Reviewer #3: Referee report for “The impact of school closure during the third wave of the COVID-19 pandemic in Italy: evidence from the Milan area”

Summary: The authors attempt to address the important question of the effect of school closures on the spread of COVID-19, focusing on the Lombardy area in Italy during an outbreak in Feb – Apr 2021. The authors concentrate on children ages 3 to 11, whose schools were shut down on March 5th until April 5th. Older students’ schools had already been shut down for a couple of weeks by the time the 3-11 year-olds’ schools were closed. The authors estimate Poisson regression models for COVID cases for 3-11 year-olds and 20+ year-olds separately (also 12-19 in an appendix), pursuing an interrupted time-series research design, allowing for a slope break 6 days after the schools closed. They find that after the schools shut down, case counts decreased by about 4% for 3-11 age group and 3% for the 20+ group. While the authors do not test for differences between estimates in these two groups, it seems quite clear from their results and plots that these 4% and 3% estimates are not statistically distinguishable. I would argue they represent the same effect. The authors then conclude that, “it is nonetheless clear that school closure contributed substantially to mitigate the spread of the Alpha variant among the population” (p. 15).

Comments

1) I think the authors address an important question and they have some nice data on covid cases by age in the Milan, Italy area.

2) That said, I do not think the authors employ a credible research strategy in this case with their interrupted time series. This is an obsolete strategy and should be replaced with more modern approaches. In the authors’ case, a difference-in-differences (DD)/event study approach would be a modern and credible research design that I think could be feasible for the authors given their data. The problem with the interrupted time series approach is the authors can’t account for other factors that could be correlated with time. Other COVID-19 precautions implemented at the same time as the school closures are one of these other factors. The authors discuss these and clam they are likely to have little effect (without evidence). They might have a point, but that doesn’t mean that we shouldn’t use available methods to be more confident, rather than just having to take the authors’ words for it. More important, though, is the fact that populations respond strongly to COVID-19 out-breaks on their own, without the government having to force them to change their behavior. We have seen this in many ways during the COVID-19 pandemic. Thus, while the government responds to the covid outbreak, so do the people in changing their behavior. This change in behavior could be the driving factor for the fall in cases (or a contributor). The authors’ interrupted time series does not allow for any way to control for these factors.

3) The authors would be much better trying to provide a control group for comparison, as this group would control for the behavior of the population, the other government restrictions, and any other omitted factors. Looking at their data, it seems like the authors could have used 12-19 year-olds as such a group, or even the 20+ year-olds, though 12-19 is the better group as most of that group also attends school. The authors note in Table 1 that with entry into the “yellow zone” on February 1, schools at the second level of secondary school and above (i.e., 12 year-olds and up) had their schools closed. So why not compare cases among this group to those of 3-11 year-olds who schools weren’t closed? The 3-11 age group would control for population response to covid and also for the other government restriction that were imposed. The difference in cases between the groups could then more credibly be called the effect of school shut-down. They could even use variation from when the younger kids’ schools were shut down as part of this approach, as well. Alternatively, the authors could use the 20+ year-olds as a control group. This is not as strong a control group since these people don’t attend school, but it is far better than the approach the author currently use where there is no control for population response at all.

4) Looking at the authors’ data, we can informally perform a sort of DD approach by comparing how case rates changed over time across these three groups. The results are presented in Table 2 and in Table S2 (in the appendix). While the authors don’t formally test for differences, it is clear that the estimates of spread in the authors’ pre-period (estimate for variable “time”) are almost quantitatively identical. Moreover, during the post-period, they are still almost quantitatively the same. Without a doubt, across the groups these estimates would not be distinguishable statistically. Thus, in my view the authors’ data is suggestive of their being no effect of the school shut-downs. That said, if the authors had performed a more formal DD style analysis, maybe they would have been able to show a difference. But at this point, I see their analysis as suggesting no effect.

5) Given the above, I think the authors’ conclusions are far too strong, and are not supported by the data they present. In particular, I think the following statement made by the authors is unsupported by their data: “Despite this, the claim that ‘schools are safe from COVID-19’, which was widely spread among the Italian media, appeared not to be supported by the empirical evidence.” (p. 16) (After reviewing their evidence, I am *more* inclined to agree that there was no impact of the school closures and that schools are safe). Additionally, the conclusion I noted in the summary, “it is nonetheless clear that school closure contributed substantially to mitigate the spread of the Alpha variant among the population” (p. 15), is also not supported by the authors’ data. I would suggest the authors remove these statements, and any other similar ones.

Minor comments

a) Would have been helpful if the draft had included page numbers. My page numbers are the page in the PDF file I was provided.

b) The authors’ regression equation should have an exponential function in it given that it is supposed to be a poisson model, and should also have an error term.

c) P. 13: “Bearing in mind the average incubation period of the virus, we tested each model with a cut-off established at six days from the intervention date.” The authors could be clearer about what they mean by this. For a while as I was reading I thought it meant the authors lagged the data by 6 days.

d) Figure 2: I really don’t see a point to including the red-dashed lines. They just cause confusion. Plus I don’t think the authors ever really explain what it is.

e) Was school actually closed on March 5? It is not clear from the manuscript. That was a Friday, and though I am not sure if Italy follows the same system as the USA, but if schools were closed for Saturday and Sunday, maybe the authors should not consider the school closures to be implemented until March 8, the Monday in which might have been closed for the first time?

f) P. 14: “Results are adjusted for seasonality and time-varying confounders, meaning that the effects detected are not attributable to concurrent factors such as the underlying scenario applied, the epidemic trend in the whole country, or the number of swab tests performed locally or nationally.” – It is helpful to include these controls, but I think this statement is entirely too strong. One could argue that these controls are not sufficient to fully control for these factors (and could do so quite effectively, I might add), so I would not say these factors could not have *any* effect on the authors’ results. I would say that they include these controls which help address concerns that these factors could be the true causes of the authors’ estimates.

g) It is odd the way the authors at times reference the 12-19 age group yet don’t include their results in the paper (only an appendix).

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

25 May 2022

Dear Reviewers,

We thank you for your comments and suggestions, which we believed have helped us in reaching a higher quality output. We provide point-by-point answers to each comment in the file attached.

Overall, we made efforts to make our findings more comprehensible to the readers, as we believe that in the first version we did not stress enough the importance of the temporal dimension in suggesting the relevance of the school closure intervention. It is not so much the magnitude of the coefficients that convinced us about the effectiveness of the intervention, but rather the fact that the contagion curve started to decrease first in those directly affected by the intervention, and only subsequently in the adult population, which benefited from the reduction in contagion in school-aged children, only marginally affected by the virus before the novel “British” strain.

Despite we included several time-varying confounders, we agree with the reviewers that this might not be enough to completely separate the effect of the intervention from other unmeasured concomitant variations, and then that the decrease in the contagion curve could be attributable to factors other than the school closure. However, the temporal order of contagion decrease in different age groups is a very relevant point in favor of the effectiveness of the intervention, as we found no other explanation for this robust and meaningful evidence.

Hence, we deeply revised to manuscript to highlight this issue.

Yours sincerely,

The authors

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

30 Jun 2022

The impact of school closure intervention during the third wave of the COVID-19 pandemic in Italy: evidence from the Milan area

PONE-D-21-36674R1

Dear Dr. Russo,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Hao Xue

Academic Editor

PLOS ONE

4 Jul 2022

PONE-D-21-36674R1

The impact of school closure intervention during the third wave of the COVID-19 pandemic in Italy: evidence from the Milan area

Dear Dr. Russo:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Hao Xue

Academic Editor

PLOS ONE