Impact of implementation timing on the effectiveness of stay-at-home requirement under the COVID-19 pandemic: Lessons from the Italian Case.

When a new infectious outbreak emerges, governments must initially rely on non-pharmaceutical interventions (NPIs) to mitigate the impact of the pathogen. Although a strict stay-at-home requirement (i.e., lockdown) presents high effectiveness in reducing patients hospitalized in intensive care units (ICUs), it comes with unintended physical, psychological, and economic damages for the citizens. Using how Italy managed the COVID-19 outbreak from February to September 2020 on a national basis, this study aims at understanding the impact of implementation timing on the effectiveness of NPIs. Our findings may be helpful to avoid the implementation of stay-at-home requirements when it is not strictly necessary. A compartmental SEICRD model was developed to create the baseline scenario without NPIs. Generalized Poisson regressions were applied to study the change in effectiveness over-time of NPIs on Avoided ICUs for each one of the Italian regions. Our study suggests that although the stay-at-home requirement is the most effective measure in reducing ICU hospitalizations in regions encountering the outbreak early, its effectiveness decreases in regions encountering the outbreak later, where a set of other NPIs are more effective. We developed a reference of daily new cases when lockdown should be implemented or avoided, accordingly. Our findings could be useful to support policymakers in contrasting the pandemic and in limiting the societal and economic impact of stringent NPIs.

Introduction

On December 31, 2019, the Wuhan Municipal Health Commission reported to the WHO (World Health Organization) an outbreak of pneumonia, later identified as a novel coronavirus, clustered in and around a seafood market in Wuhan, China [1]. Over two years later, SARS-CoV-2 has affected 224 countries and territories across the World with 448,251,362 officially recorded cases and 6,0027,902 fatalities [2].

Mass vaccinations appears to be the potential solution to slow down and eventually interrupt the chain of infection once herd immunity is reached and more importantly to reduce the number of hospitalization, critical cases and deaths [3], [4], [5], [6], [7], [8]. The advancement carried by the mRNA technology showed that when facing a global emergency, with coordinated resources, vaccines development can be supplied to the population in a record time [9]. However, logistic issues [10], vaccine hesitancy [11,12], or merely the time necessary for the development of the vaccine itself, make the role of non-pharmaceutical interventions (NPIs) essential in containing the spread of the contagion, especially in the early phase of the epidemic.

The appearance of novel human pathogenic viruses as well as the re-emergence of known viruses is increasing, mainly caused by increased population density, imports of non-autochthonous animals and plants, deforestation, and uncontrolled urbanization [13]. More recently, the correlation between climate change and emerging disease has been investigated, showing a link between the phenomenon [13]. Aware of the threats of future epidemics, deriving conclusion from the current COVID-19 pandemic is fundamental to not get caught off guards in case another potential virus spillover would emerge or in the eventuality that new COVID-19 variants may cause more breakthrough infections, requiring the (re)introduction of NPIs while developing and testing new vaccines.

During the last two years, governments across the world relied on NPIs to contrast the expansion of the virus. Lockdown, also known as stay-at-home requirement, has been one of the most significant and widespread NPIs implemented during the COVID-19 pandemic [14]. The implementation of large-scale extended periods of lockdown not only has resulted in significant damage for local and global economies [15,16], but also has significantly impacted the mental health of citizens, especially children and young people [17], [18], [19]. That is why, in case a novel infectious pathogen appears, it is important to understand when a strict lockdown should not be implemented in favor of a set of other NPIs.

Several studies analyzed the effectiveness of NPIs. Some authors proposed predictive models [20], [21], [22], [23], [24] based on epidemic simulations and epidemiological models to forecast the behavior of the epidemic with or without a set of NPIs [20,22]. Some others conducted cross-countries analysis to assess the effectiveness of NPIs in reducing number of COVID-19 cases or COVID-19 reproduction number (Rt) [25], [26], [27], [28], [29], [30], [31], [32]. Other research focused on a single country, forecasting the future trend of the pandemic to help plan a successful transmission control strategy. Moore at al. developed a mathematical model to estimate the risk of early relaxations of NPIs in UK [33], while Giordano et al. and Min et al. focused on the effectiveness of contact tracing and social-distancing measures at a national level in Italy and South Korea, respectively [34,35]. Italy, one of the first countries to face the rise of infections after the main outbreak in Wuhan, has been analyzed by several authors. Giordano et al. and Della Rossa et al., in particular, developed a compartmental model demonstrating that social distancing measures should be coupled with widespread testing and contact tracing [34], and that lockdown should be implemented at a regional level [36]. Scarabaggio et al. [37] after calibrating the regional pandemic dynamics with a compartmental model used a multi objective optimization function to determine the optimal NPIs strategy, minimizing the societal and economic cost. Similarly, Alfano et al. focused on the impact of school closure in reducing the spread of the disease in Italy, introducing a synthetic control method [38]. Finally, Parino et al. [39] used Italy as a test-bed for a meta-population model to assess both social distancing and mobility restrictions. The framework confirmed the importance of time and severity of the outbreak in the effectiveness of the NPIs.

Despite the numerous works published so far, to the best of our knowledge an analysis ex-post aiming at understanding the change in effectiveness of NPIs, especially ‘stay-at-home requirement’, depending on the timing of their implementation, and where the number of ICUs is used as a proxy for the severity of regional outbreaks, is still almost unexplored. Oraby et al. [40], for example, considered both timing and duration of lockdown, concluding that introducing too early or too late a lockdown may reduce its effectiveness in controlling the spread of the disease. However, their evaluation was based on the hypothetical estimate from a theoretical model rather than using an empirical investigation.

Italy offers a good empirical case to study the impact of implementation timing on the effectiveness of NPIs because despite Italian regions were staying at different phases of the pandemic, a nationwide stay-at-home requirement was imposed on March 20, 2020. Thus, using how Italy managed the COVID-19 outbreak from February to September 2020 on a national basis, this study aims at understanding the impact of implementation timing on the effectiveness of NPIs. We first developed a compartmental SEICRD model to create the baseline scenario without NPIs. Then we used generalized Poisson regressions to study the change in effectiveness over-time of NPIs on Avoided ICUs for each one of the Italian regions. Our findings not only can further validate how implementation timing significantly affects the effectiveness of NPIs, but also provide an additional scientific reference for future novel viruses.

Materials and methods

Development of the baseline scenario (SEICRD model)

Mathematical models for infectious disease modelling aim at predicting the development of the epidemic and to suggest to governments and policymakers suitable strategies for the containment of the spread of the disease [37,41,42]. A time-dependent SIR (Susceptible-Infectious-Recovered) model is a common approach for disease modelling. The SIR model consists in dividing the society in ‘compartments’, depending on the infectious status of each individual [43] and tracing how the individuals progress between the compartments. Using a compartmental model, we firstly created a framework that predicts the course of the pandemic without any NPI. The model displays the potential number of critical patients hospitalized in ICUs after contracting the infection for each region of the dataset.

Following the line of research of other papers [20,22,[34], [35], [36], [37],40,44,45], in this study we modified the original SIR model to include other compartments to better portray the dynamic spread of COVID-19, among which (i) Critical (C), (ii) Exposed (E), and (iii) Dead (D) compartments. The transition from one compartment to another is regulated by a set of differential equations (Eq.S1 in Supporting Information -SI-). The parameters of Eq.S1 are listed in Table S1 in SI, along with their values and explanation. Fig. 1 presents the transition graphs of the model. The modified SIR model will be defined as SEICRD model in the present work.

Fig. 1

Transition graph for the SEICRD model.

From the SEICRD model we obtained the baseline scenario for each region of the dataset, that represents the hypothetical course of the pandemic without any intervention (NPIs). Fig.S1 in SI graphically compares the ICUs projected by the SEICRD model (SEICRD ICUs) with the official ICU hospitalization data collected by the Italian government (Official ICUs) [46]. From the difference between the SEICRD ICUs and the Official ICUs we obtained the dependent variables of our model (Avoided ICUs) for each one of the 20 Italian regions.

Since there is not a unanimous consensus about the real parameters that can best describe SARS-CoV-2, we collected different values found in the literature to construct the model which describes the generic baseline scenario (Table S1), defined in this paper as Mod1. The parameters used to set the baseline scenario are the same for all the regions, except for population (N) and probability that an infected individual requires ICU hospitalization given that they vary on a regional base. Since there is a difference in life expectancy between Italian regions and advanced age has been proven to be a significant factor for severe and critical outcomes [47,48] we decided to calculate at a regional level as explained in Text S2, Fig.S2 and Fig.S3 in SI to address regionals heterogeneity. Official data collected from Istituto Superiore Sanità validate the effect of age on risk of ICU hospitalizations, although not adjusted by other comorbidities [49].

The basic reproduction number is one of the parameters which can affect the most the SEICRD ICUs. In epidemiology it represents the expected number of cases directly generated by one infected case and it is not a biological constant of the virus since it may be affected by both environmental conditions and the behavior of the population. Therefore, is an imprecise estimate based on assumptions [50]. According to previous studies on the effectiveness of NPIs in Italy, was between 3.49 and 3.84 [51]. We therefore considered a higher value when no NPIs are implemented. Oraby et al., for example, calculated that the of the first detected COVID-19 strain was 6.47 [40].

Hence, we selected for Mod1 the SEICRD parameters (Table S1) which led to the number of SEICRD ICUs higher than the number of Official ICUs for each region of the dataset. It is reasonable to assume that without any NPIs the development of the outbreak would have been more severe, in terms of both COVID-19 cases and, in turn, critical cases hospitalizations. Therefore, the selected set of parameters complies with this assumption.

The described methodology had been applied, using Python 3.8.2 and the ‘statsmodels’ library, to all our regional datasets to produce the baseline SEICRD ICUs (Mod1).

Setting initial data-point of the analysis (Official vs SEICRD )

When dealing with exponential growth in infections and/or critical cases and deaths, timing is fundamental. Should be noted that the first data-point considered in this analysis () differs across regions and represents the day when the first COVID-19 patient of the region under assessment has been hospitalized in ICU. The end of our study-period, instead, is the same for the entire dataset (September 30, 2020) and approximately represents the end of the first wave of COVID-19 in Italy.

In Italy, infections, hospitalizations and deaths have been recorded from February 24, 2020 at a national level [46,52]. However, considering that some regions already had on that day several critical cases in ICU (e.g., 19 critical cases in Lombardia) and the unpredictability and severity of the initial outbreak, we may expect that ICUs hospitalization started a few days earlier but were not reported due to poor testing capacity [53], limited at the beginning to only symptomatic cases [52], and a general low readiness of the Italian healthcare system [54]. Assuming that all the regions should present a similar exponential growth of cases at least for the first few days of their outbreaks, we defined as “Biased regions”, those regions whose official hospitalizations started with more than one case hospitalized in ICU (Fig.S4 in SI) as opposed to “Non-biased regions”. These last recorded their hospitalization with a more reasonable trend starting with one single hospitalization. For Non-biased regions we considered the first official hospitalization in ICU as the first data-point (Official ). Instead, Biased regions, presumably present a bias in how ICU cases were recorded in the first days of the pandemic. Hence, with the previously described SEICRD model, we identified the hypothetical first day when an ICU patient was hospitalized, defined as SEICRD (example in Fig.S5 in SI).

To validate the ability of the SEICRD model to replicate the very beginning of the exponential growth of ICU cases, we compared our Official with SEICRD for Non-biased regions (Fig.S6 in SI). Since Official and SEICRD basically coincide (Fig.S6 in SI), we are confident to affirm that the SEICRD model is accurate in capturing the dynamics of the early phase of the pandemic and therefore in our analysis we considered SEICRD for Biased regions.

Regression analysis between the avoided ICUs and NPIs

We applied regression analyses to study the effectiveness of NPIs that are workplace closure, close public transport, stay-at-home requirement (i.e., lockdown), restriction on internal movement, and facial covering in reducing the number of critical cases requiring hospitalization in ICU (i.e., Avoided ICUs equal to SEICRD ICUs minus Official ICUs) for every region of the dataset. Since our aim is to understand the change in effectiveness of NPIs, specifically the stay-at-home requirement, depending on the implementation timing, we chose NPIs whose severity index changed during the observation period. The effectiveness of NPIs whose severity indexes do not change over time may be difficult to compute, leading to misleading interpretations of the regressions’ results. Other NPIs (e.g., school closure) presented high multicollinearity and had been removed accordingly. Lastly, no-significant variables were omitted as well.

Several authors used regression models to assess the correlation between NPIs and the course of the pandemic [25,29,31,32,52,55]. Text S3 in SI lists the model used in those studies. Poisson regression is the most suitable for infectious disease models, often used by epidemiologists, especially with counts based datasets [56,57]. More precisely, given the nature of the data, we used a generalized Poisson Consul model, defined as GP-1 [58]. This model is more appropriate when the dataset is over-dispersed and it is necessary to model count data with a long right tail [59]. Eq.S2 in SI presents the GP-1 model between the Avoided ICUs and several NPIs.

All the independent variables range from 0 to 1 based on the strictness of the implemented measure (0 minimum, 1 maximum; Table 1 ). The data has been collected from the “Coronavirus Government Response Tracker” [60] from Oxford University. Fig.S7 in SI displays when the measures have been implemented at a national level, in comparison with the development of the pandemic, in terms of ICU hospitalizations, in Italy.

Table 1

The independent variables of NPIs collected from the Oxford dataset.

ID	Description	Values
k1	Workplace closure	0 - no measures0.33 - recommend closing 0.67 - require closing for some sectors or categories1 - require closing for all-but-essential workplaces
k2	Close public transport	0 - no restrictions0.5 – significantly reduce volume/route/means of transport1 – require closing
k3	Stay-at-home requirement (lockdown)	0 - no measures0.33 - recommend not leaving the house0.67 - require not leaving the house with exceptions for daily exercise, grocery shopping, and 'essential' trips1 - require not leaving the house with minimal exceptions
k4	Restrictions on internal movement between cities/regions	0 - no measures0.5 – recommend not to travel between regions/cities1 – internal movement restrictions in place
k5	Facial coverings	0 - no policy0.25 - recommend0.5 - require in some specified shared/public spaces0.75 – required in all shared/public spaces1 – required outside the home at all times

With this methodology, we created a dataset for every region, with the daily Avoided ICUs and the severity index of each NPIs starting from the first day in which a critical patient has been hospitalised (Official for Non-biased regions and SEICRD for Biased regions) until September 30, 2020, collecting more than 250 daily observations for each region.

To easily compare the results of the regression models, we introduced a ratio defined as “Relative Contribution (RC)” to evaluate the contribution, in percentage, of each NPI to Avoid ICUs (Eq.S3 in SI). With this ratio we therefore evaluated the relative contribution of the NPIs that were effective in reducing ICUs without considering in the equation NPIs with negative impact on Avoided ICUs.

Following other previous papers [25,29,30,61,62], we assumed that a NPI takes a few days before showing any type of results. In fact, incubation period of COVID-19 is around 5 days while approximately 7-11 days are necessary for the development of symptoms [63,64]. This means that a significant amount of positive cases could emerge in the days following the implementation of social distancing measures and NPIs can therefore take days after the implementation to show an effect [30,38,45,52]. Alfano and Ercolano [30] and Gatti and Retali [45] suggest that lockdown takes 10 days to show its effect, while Baier et al. showed that NPIs on average may take 2 weeks before being effective [61]. Lockdown should last for at least 20-30 days; a significantly longer enforcement of this strict NPIs does not seem to provide further benefits in controlling the virus [30,40]. More in general, it appears that both introduction of NPIs and their removal present a delayed-effect on the pandemic's dynamic which is in line with the incubation period of the disease and the days needed for an infected individual to develop symptoms [38].

Thus, to address the delayed-effect of NPIs we also conducted additional regression analyses between the Avoided ICUs and the measures of NPIs stringency simulating also a 7- and 14-days lag effect in the NPI effectiveness.

We created a total of 3 regression models (3 lag effects) for each of the 20 regions, and a total of 60 GP-1 models. Moreover, we ordered the regions based on Official for Non-biased regions and SEICRD for Biased regions, thus creating a temporal rank of regions. There is almost a month between the first critical case in Lombardia and the one in Sardegna, last region to record the first ICU hospitalization. This means that each region was in a different time-point of its respective pandemic curve when NPIs have been applied at a national level and the effectiveness of the measures (especially k3: stay-at-home) vary by regions, depending on their time-point of the pandemic curve.

Sensitivity analysis of the impact of SEICRD parameters on the effectiveness of NPIs

To validate our study, we performed a sensitivity analysis on the parameters of the SEICRD model. If the results of our regression models between Avoided ICUs and NPIs are coherent despite a change in the parameters used in the SEICRD model, it means that the study is reliable, and it does not strictly depend on the parameters’ selection, on condition that, the magnitude of the pandemic without NPIs would have been more severe than the actual outbreak (as mentioned in the description of the SEICRD parameters). This means that for each data point the number of SEICRD ICUs is higher than actual recorded data of ICUs for each region. We therefore selected a region (i.e., Toscana) and performed the sensitivity analysis, changing each parameter (i.e., δ, γ, R) by for a total of 16 different projected Avoided ICUs for the region under analysis as shown in Fig.S8 in SI.

Once obtained the SEICRD ICUs for Toscana, we performed other 48 regression models (16 additional Avoided ICUs resulting from the sensitivity analysis 3 lag effects).

Results

The impact of NPI ‘k3’ (stay-at-home requirement) on Avoided ICUs

Firstly, we checked our regional datasets, and we discovered that are over-dispersed, confirming our hypothesis to use the GP-1 model. We trained and tested the regression models to visually check the goodness-of-fit of the model and the quality of the predictions. For comparison, we run the analysis with other two typologies of regression models: (i) standardized Poisson and (ii) multilinear regression. GP-1 shows the best goodness-of-fit within the tested models (Text S4).

Fig.S9 in SI shows that the GP-1 model is reliable and can predict the daily Avoided ICUs leading us to affirm that the coefficients’ estimators are reliable.

We therefore considered the change in the relative contribution of each coefficient estimator as a proxy for the effectiveness of the NPIs in reducing COVID-19 critical cases hospitalizations. We specifically focused on the change in effectiveness of the stay-at-home requirement (k3) throughout the temporal rank of regions.

The relative contribution of k3 (stay-at-home) is higher in Lombardia, Veneto and Emilia Romagna, the first regions that have been hit by the COVID-19 outbreak in Italy, but the lockdown's contribution decreases as the regions encountered the COVID-19 later. This finding is consistent across the models (Fig.S10 in SI), with an R2 ranging from 0.49 (lag 7) and 0.67 (lag 0) between the change in Relative Contribution and the temporal rank of regions. This result is consistent with the literature; Xiu et al. found that NPIs produce effects within 7 days [62]. The model with 14 days of lag, instead, does not show a significant correlation between the change in effectiveness of k3 and the temporal rank of the regions. With 14-days delay in the NPIs effectiveness, lockdown always has RC > 0.5.

Notably, Fig.S10 in SI highlights two ‘outlier’ regions, namely (i) Valle d'Aosta and (ii) Molise. Although our model cannot provide an univocal empirical explanation, there are several factors potentially causing this result. Firstly, these are the least populated regions in Italy (i.e., 125,034 inhabitants for Valle d'Aosta and 300,516 for Molise, Fig.S11 in SI). Number of Official ICUs and COVID-19 new cases is significantly lower respect to other regions (peak of ICUs hospitalizations: 9 Molise; 1381 Lombardia), resulting in less accurate results. Less testing capacity and less prepared regional healthcare systems may also have caused a delayed detection of COVID-19 cases. This last hypothesis can be explained by the rate of increase of ICU hospitalizations, higher than other regions with more cases and more reliable data (Fig.S12 in SI compares the trend in ICUs hospitalizations in Valle d'Aosta, Molise, Veneto and Lombardia). Removing the two outliers significantly increase the R2 between the change in the Relative Contribution of the stay-at-home requirement and the temporal rank of the regions: lag0 from 0.67 to 0.82; lag7 from 0.49 to 0.74; lag14 from 0.14 to 0.21 as shown in Fig. 2 .

Fig. 2

Change in the Relative Contribution of stay-at-home requirement (k3) throughout the temporal rank of regions – outliers removed. The x-axis represents the time rank of the datasets (regions) based on Official for Non-biased regions and SEICRD for Biased regions, starting with Lombardia until Sardegna. The y-axis represents the Relative Contribution of the ‘stay-at-home’ requirement. Subplots (a), (b), (c), represent the change in the Relative Contribution of NPI k3 considering a lag of 0, 7, and 14 days in the effectiveness, respectively. Subplot (d) plots the three lags on the same graph for comparison.

Notably, with a lag of 14 days, k3 would have been effective only after the peak of hospitalizations (March 31, 2020). We hypothesized that the effectiveness of lockdown in reducing ICU hospitalizations decreases if the NPI is implemented too close/after the peak. However, a more in-depth analysis should be conducted to validate this observation.

To further validate our assumption to use SEICRD instead of Official in Biased regions, we also run our model with Official for all the region (both Biased regions and Non-biased regions), obtaining similar but slightly less significant results (Fig.S13 in SI). Nevertheless, this confirms the reliability of our model to produce consistent conclusions.

The impact of other NPIs on Avoided ICUs

We investigated the change in the effectiveness of the other NPIs introduced by the Italian government. The impact of NPIs on the Avoided ICU cases is statistically significant among all the regression models. The level of multicollinearity among the NPIs is acceptable as shown by the heat map in Fig.S14 and Text S5 in SI.

Besides the stay-at-home requirement, k2, k4, and k5 are effective in the reduction of critical case hospitalizations, albeit to a different degree. k1 (workplace closure) present in every model a negative contribution to the reduction of cases in ICUs. Despite the legislative decree was issued at a national level in the same day, the implementation varied over the regions without clear guidelines. Some workers initially kept working until clearer indications were provided. It is plausible that a percentage of these workers contracted the virus and then infected their relatives once required to work from home. However, this hypothesis cannot be proved by the present model and therefore we did not consider this variable in our study.

As opposed to k3, instead, the other NPIs contribute to more Avoided ICUs in the regions encountering the COVID-19 later. Fig.S15, Fig.S16, Fig.S17 presents the change in effectiveness of the NPIs k2, k4, and k5 along the temporal rank of regions, considering a lag of 0 (lag0), 7 (lag7), and 14 (lag14) days in the effectiveness of the measures, respectively. This means that for regions affected later by the pandemic, the lockdown was not the best solution since the number of cases was lower and thus, a set of alternative NPIs would have been more effective, in line with the results of other studies [32]. On the other hand, Lombardia was already in a difficult situation when the Italian government decided to introduce more stringent NPIs. In that case, most effective solution to “flatten” the pandemic curve was the strict stay-at-home requirement, although the draconian measure was implemented at a national level. Regions that show a “delay” in the pandemic curve, according to our model, could maintain under control the pandemic focusing on other measures.

Particularly, k4 and k5 show an increasing effectiveness over the temporal rank of regions (R2 up to 0.77) (Fig.S15, Fig.S16, Fig.S17 in SI). k2 (close public transport), instead, appears to maintain a stable RC across regions and lags models, resulting the NPI with highest RC after lockdown with lag 14 (Fig.S16)

Table S2, Table S3, and Table S4 in SI summarizes the Relative Contribution of all the considered NPIs for the 20 regions of the dataset and the 3 lags hypothesized.

The impacts of SEICRD parameters on the effectiveness of NPIs

The result of the regression models validates our hypothesis that the parameter selection of the SEICRD model does not significantly affect the result of the study, which is coherent throughout all the regions without being significantly affected by different parameters of the SEICRD model. Fig.S18, Fig.S19, and Fig.S20 in SI present the results of the sensitivity analysis conducted in Toscana with lag0, lag7 and lag14, respectively. Among the NPIs considered, the parameter selection has a minimum impact on k2 and k4 while a stronger impact on k3 and k5, especially with lag0 (Fig.S18). Fig.S21 focuses on k3 and k5, showing that R is the most influential parameter.

Use of the number of daily new cases to infer the effectiveness of k3 versus other NPIs

Since there is a certain time-interval from onset of symptoms and the eventual hospitalization for critical conditions, ICUs are a delayed image of the pandemic. Moreover, the probability of being hospitalized in ICU due to covid is not homogenously across the population. An outbreak can firstly occur within a low-risk population, such as children and only subsequently spread across other age groups more at risk. For this reason, we cannot use ICUs as a good indicator to introduce or remove the stay-at-home requirements or other NPIs.

We collected the weighted-population average number of recorded cases in the days in which the strict stay-at-home requirement (k3=1 on March 20, 2020 [65]) has been implemented at a national level for the first three regions that presented a relative contribution of k3 lower than 50% (Fig.2). This number will be the reference for implementing a strict stay-at-home requirement. Regions showing a contribution of k3 lower than 50% could have chosen a set of other NPIs instead of the draconian measure. Notable, with a lag of 14 days in the effectiveness of NPIs, k3 presents a RC ranging from 53.96 to 74.19. Hence, we selected the last three regions in the temporal rank of the regions (with lowest Relative Contribution of k3) to set our reference value.

Considering that the median incubation time of COVID-19 is 5.1 days, we collected the 5-days (March 18,19,20,21,22, 2020) average of new cases recorded for the selected regions and calculated the incidence new cases/1,000,000 as shown in Table 2 . A lockdown should be considered when the 5-days moving average of new cases is approximately 107/1,000,000 inhabitants for lag0, 70/1,000,000 inhabitants for lag7 and 43/1,000,000 inhabitants for lag14.

Table 2

Reference for the implementation of k3, stay-at-home requirement. Thresholds for the implementation of the stay-at-home requirement (k3 = 1). The column “New cases/1,000,000 inhabitants” shows the reference for lag0, lag7, and lag7.

Region	Lag	New cases recorded (5-days average)	New cases/1,000,000 inhabitants (95% confidence interval)
Marche; Piemonte; Toscana	0	210; 504; 245	107 (87-128)
Trentino; Sicilia; Calabria	7	191; 79; 32	70 (29-112)
Friuli Venezia Giulia; Basilicata; Sardegna	14	96; 12; 44	43 (26-60)

Discussion

Our study presents several limitations. Firstly, our references for the introduction of a stay-at-home requirement are calibrated based on the Italian situation. On one hand, this allowed us to easily compare regional datasets since data were collected in a similar way. On the other hand, this limits the sample size of our study, and therefore, the accuracy of the references of the number of daily new cases for the implementation of k3 found. With a cross-country study we could obtain more precise references, but without a standardized testing policy and tally of cases, comparing different countries’ outbreaks during the first wave of COVID-19 cases would have been challenging.

Secondly, we could not include in our analysis all the NPIs introduced by the Italian government due to multicollinearity problems or simply for the impossibility of statistically evaluate their effectiveness over time. However, further studies may apply this methodology to other countries to confirm our results and provide new findings regarding other NPIs effectiveness.

Thirdly, there are several behavioral, societal, economic, institutional, and infrastructural factors influencing not only the response of a region to a healthcare emergency but also the complains with NPIs of citizens [38,44,52,66]. Italy presents a strong heterogeneity within the country that should be considered by policymakers when evaluating the most effective and efficient set of policies [52]. Compartmental models tend to oversimplify the dynamics of the pandemic and those differences are generally not included in the modeling and are also out of the scope of the present study. Despite this, in the context of this work, we partially addressed regional heterogeneity including demographic data (particularly regional age-distribution), since our focus is on critical hospitalizations, and age is one of the variables that influence the most severe outcomes from COVID-19 infection.

Lastly, seasonality played a role in the trajectory of the pandemic during the first wave of COVID-19 in Italy [67,68]. Italy presents differences in terms of temperature and humidity throughout the peninsula, but since the entire country is within the Northern Tropic and more precisely the Mediterranean area, we omitted the seasonal component at a regional level in line with other studies [36]. However, this variable should be considered in cross-country/continental analysis.

Nevertheless, our study confirms that the implementation timing of NPIs (especially lockdown) significantly affects the effectiveness of the policies [32,36,39,40] and should be carefully considered by policymakers. Moreover, it shows the importance of the estimation of the pandemic peak and the beginning of the pandemic [40]. Although our study presents different assumptions and was calibrated only on the Italian case, it is noticeable that the model's results can validate the time intervals suggested by Oraby at al [40]. This last paper suggests that (despite differences among countries) the effectiveness of lockdown is relatively small if implemented 30 days before the peak of the pandemic projected, with no NPIs, higher if implemented around 15 days before the peak, and maximum when closer to the peak (around 5 days), similarly to what emerged from our research (Text S6 in SI).

We need to point out that our references for the introduction of the stay-at-home requirement should not be considered as absolute values applicable to every context. First, the reference was determined based on its effectiveness of reducing the overloading of ICUs. There are other side effects of a strict lockdown policy for consideration. The strict lockdown implemented in Italy has severely affected screening programs leading to a higher number of deaths from cancer and cardiovascular problems [69,70], as also pointed out by Haug et al [32], as well as many other aspects of citizens’ life, including, inter alia, phycological health risks, economic vulnerability, and domestic violence [71]. Governments must consider a tradeoff to counterbalance benefits and drawbacks of NPIs, considering all the unintended collateral damages resulting from the implementation of stringent measures. A strict stay-at-home requirement is effective but should be introduced only when inevitable. Regions or cities which do not present an uncontrolled and severe outbreak, must firstly rely on less radical NPIs [39]. Moreover, long periods of lockdowns may reduce the confidence of citizens in governmental decisions [72], leading in turn to a generic distrust which may result in a reduced compliance with other less intrusive NPIs, hindering the efforts spent.

Second, the references themselves are based on the number of new cases recorded by the Italian government, which was, especially at the beginning of the pandemic, a biased indicator. Testing capacity, and testing policy significantly influenced the number of new cases recorded during the initial outbreak in Italy. Therefore, it is reasonable to consider a much higher number of hidden cases, especially considering the contribution of asymptomatic carriers, as subsequently proved by serological tests [73]. Our reference should probably be higher if all the cases were tracked.

Moreover, in recent waves, the high percentage of population vaccinated, and the reduced severity of Omicron variants reduces the hospital admission and death [74]. Both the ICUs without and with NPIs in recent waves could be very different from the first wave if a strict lockdown was implemented nationwide. As the severity drops, our references of new daily cases for implementing the strict stay-at-home should be higher. In our study, the ICUs with NPIs was based on empirical observations. In the future, by developing an accurate pandemic prediction model to estimate the Avoided ICUs without and with NPIs, our analytical framework can be generalized to provide an up-to-date estimate of the reference for implementing a strict lockdown and other NPIs.

Conclusion

A compartmental SEICRD model and generalized Poisson regressions models were used to evaluate the effectiveness of the stay-at-home requirement and other NPIs in reducing critical patients hospitalized in ICUs during the COVID-19 pandemic in Italy. Our findings suggest that the NPIs close public transport, stay-at-home requirement, restriction on internal movement, and facial covering are effective in the reduction of critical cases hospitalized in ICUs to a varying percentage depending on the phase of the pandemic. Workplace closures presents the opposite result. The stay-at-home requirement is the most effective measure to reduce critical cases in the regions encountering the COVID-19 earlier, while its effectiveness decreases in the regions encountering the COVID-19 later. On the other hand, other NPIs become more effective in those regions that recorded later their first patient hospitalized in ICU. We developed a general reference number of daily new cases when the stay-at-home requirement should be implemented or not. Our reference could be useful to support policymakers in the decision to contrast the pandemic limiting the societal and economic impact of the draconian measure. Remarkably, despite the contribution of a set of NPIs to the mitigation of the pandemic, our model underlines that, with an uncontrolled high daily number of new recorded cases recorded in the first phases of the pandemic, the most effective solution is the introduction of strict stay-at-home requirement along with strict social distancing measures to flatten the pandemic curve. However, the measure should not be applied at a national level, but localized, depending on the regional outbreaks. We used this reference number to interpret the decisions of the Italian government in controlling the COVID-19 pandemic. Our findings are limited in terms of data availability and sample size but presents a methodology that can be applied to further potential outbreaks of new pathogens, providing an additional tool to support the policy decision making process and avoid the collapse of healthcare systems.

Data and materials availability

All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. All de-identified data related to this paper may be requested from the corresponding authors.

Funding

The work was supported by the Hong Kong Research Grants Council - PhD Fellowship, the Guangdong Basic and Applied Basic Research Foundation, and the start-up support from the Hong Kong University of Science and Technology. The views and ideas expressed here belong solely to the authors and not to the funding agencies.

Author Contribution

Each named author has substantially contributed to conducting the underlying research and drafting this manuscript. S.M. designed the study, conducted the data analysis, and wrote the manuscript. Z.L. designed and directed the study and provided substantive revisions of the original manuscript.