Quantifying the Effect of Public Activity Intervention Policies on COVID-19 Pandemic Containment Using Epidemiologic Data From 145 Countries.

OBJECTIVES: Most countries have adopted public activity intervention policies to control the coronavirus disease 2019 (COVID-19) pandemic. Nevertheless, empirical evidence of the effectiveness of different interventions on the containment of the epidemic was inconsistent.
METHODS: We retrieved time-series intervention policy data for 145 countries from the Oxford COVID-19 Government Response Tracker from December 31, 2019, to July 1, 2020, which included 8 containment and closure policies. We investigated the association of timeliness, stringency, and duration of intervention with cumulative infections per million population on July 1, 2020. We introduced a novel counterfactual estimator to estimate the effects of these interventions on COVID-19 time-varying reproduction number (Rt).
RESULTS: There is some evidence that earlier implementation, longer durations, and more strictness of intervention policies at the early but not middle stage were associated with reduced infections of COVID-19. The counterfactual model proved to have controlled for unobserved time-varying confounders and established a valid causal relationship between policy intervention and Rt reduction. The average intervention effect revealed that all interventions significantly decrease Rt after their implementation. Rt decreased by 30% (22%-41%) in 25 to 32 days after policy intervention. Among the 8 interventions, school closing, workplace closing, and public events cancellation demonstrated the strongest and most consistent evidence of associations.
CONCLUSIONS: Our study provides more reliable evidence of the quantitative effects of policy interventions on the COVID-19 epidemic and suggested that stricter public activity interventions should be implemented at the early stage of the epidemic for improved containment.

Introduction

As of July 2021, the severe acute respiratory syndrome coronavirus 2, causing the coronavirus disease 2019 (COVID-19), is still spreading globally. The nonpharmaceutical interventions appear to be an important way that could reduce virus transmission until effective treatment regimens or mass immunizations are available. , These intervention strategies include swift surveillance, quarantine, and physical distancing measures such as school and workplace closing, internal and external travel restrictions, and stay at home requirements.4, 5, 6, 7 Almost all countries have adopted a series of containment and closure policies at different time points, and some seem to have curbed the virus transmission with varying degrees of success.8, 9, 10 Nevertheless, the quantitative evidence on the effectiveness of different intervention policies has been inconsistent.

Most early studies applied modeling assumptions using data within a single country to examine the effectiveness of interventions. , 9, 10, 11, 12, 13 For example, Lai et al developed a simulation framework using daily travel networks across China. They estimated that without nonpharmaceutical interventions, the COVID-19 cases would likely have increased 67-fold. Another research group developed an age-structured susceptible-exposed-infectious-removed model with data from medium-sized cities in the United States, suggesting that interventions that started earlier in the epidemic delayed the epidemic curve and interventions that started later flattened the epidemic curve. Nevertheless, those results were derived from mathematical assumptions under presumptive scenarios and thus could not be verified.

Data from individual countries suffered from its intrinsic incapability in quantifying and comparing the effects of different interventions. By now, evidence from comparative analysis using data from multiple countries is still inconsistent. The major issue in analyzing the causal relationships is that there exist unobserved confounders such as different testing capacities over time and heterogeneity across countries. Previous empirical studies leveraging straightforward statistical methods failed to address this problem. , By early July 2020, with most Asian and European countries reaching the late stage of the first epidemic wave and the availability of detailed intervention information, we were able to retrospectively scrutinize the effects of interventions using more sophisticated statistical methods. In this study, we performed a comprehensive analysis of the effectiveness of the timeliness, stringency, and duration of 8 public intervention policies on COVID-19 containment using data from worldwide countries. Moreover, we introduced a novel counterfactual estimator based on the time-series COVID-19 epidemic data to quantify the effects of different interventions with less bias.

Methods

Data Sources and Selection

The daily confirmed cases for COVID-19 of each country were retrieved from https://ourworldindata.org from December 31, 2019, to July 1, 2020. The data were collected and reported by the health authority of each country. The country-based time-series data for the containment and closure policies were retrieved from the Oxford COVID-19 Government Response Tracker (https://github.com/OxCGRT/covid-policy-tracker) during the same time period. Details of the data collection and annotation have been described in a working article. In brief, a group of policy and government experts routinely collected information on public policies worldwide, including containment and closure interventions, and economic and healthcare supports. The policies of our interest were containment and closure interventions including 8 regimens, namely, school closing, workplace closing, public events cancellation, restrictions on gatherings, public transport closing, stay at home requirements, restrictions on internal movement, and international travel controlling. Each of the 8 interventions was recorded on an ordinal scale representing the level of strictness of the policy. Take workplace closing for example, 0 represents no measures; 1 represents recommending closing; 2 represents requiring closing for some sectors or categories of workers; and 3 represents requiring closing (or work from home) for all-but-essential workplaces. We selected countries with a time series longer than 90 days and the number of cumulative infections > 100 to reduce uncertainties. We coded each of the 8 interventions into 3 independent variables (Start-Date, Stringency, and Duration). Start-Date was recorded as the days of intervention commencement relative to the first 100 cases occurrence in that country, which served as an indicator for timely response to the pandemic. We selected the date of the first 100 cases instead of the first case as the start point because the detection of the first case was subject to more randomness that might introduce substantial noise in the following analysis. Stringency was measured as the average level of strictness across all days during a certain epidemic phase. Duration was measured as the number of days under intervention divided by the number of total days during a certain epidemic phase.

Outcomes

The first outcome was the estimated cumulative infections per million population for each country on July 1, 2020. This outcome was used to investigate the correlations with different intervention variables. We chose July 1, 2020, as the endpoint because most Asian and European countries had reached the end of their first epidemic waves by late June or early July 2020 (see Appendix Fig. 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007). During this period, some governments lifted policy restrictions and the public loosened precautions, which in certain cases led to the resurgence of cases and deaths subsequently. Then, the governments reimposed policies in a policy seesaw as the epidemic waxed and waned. , The lift and reimposition of policies and resurgence of the epidemic would affect the estimation of intervention effects. We aim to study the effect of policies within a single epidemic wave, with the consideration of reducing confounding and uncertainty. Nevertheless, there is no precise and consistent cutoff date for all countries. Therefore, we performed additional sensitivity analyses, by altering the cutoff date 1 month earlier (June 1, 2020) or later (August 1, 2020), to test the robustness of the results. The number of reported cases represents a straightforward metric reflecting the severity of the pandemic and has been widely used in previous articles for comparison between countries. , Nevertheless, the metric has limitations because the testing and reporting strategies are different across countries. In the current study, we chose the number of true infections rather than the number of reported cases as the metric for comparison. Given that the true infections were not known, we referred to the age-structured susceptible-exposed-infectious-removed model (https://github.com/mrc-ide/squire) developed by Imperial College London to estimate true infections. This Imperial College London model is among the most widely used approaches for estimations of true infections and has been recommended by Our World in Data for studying policy effects. In brief, the model fit data on confirmed deaths by using an estimated infection fatality rate to “back-calculate” how many infections would have occurred over the previous weeks to produce that number of deaths. It also accounted for mobility and testing rates data by country if available under a range of assumptions and epidemiological knowledge to generate a less biased infection estimate.

The second outcome was the time-varying reproduction number (R ) for each country on each day. R is a measurement that represents the mean number of secondary cases that were infected by 1 index case. We used the median R estimates from the widely used EpiForecasts model (https://epiforecasts.io). The process of estimation was based on confirmed cases and deaths while accounting for uncertainties of the incubation period, the infection-to-confirmation delays, and the infection-to-death delays. The method of calculating R has been detailed in Cori et al. In brief, the transmission rate of COVID-19 can be estimated by the ratio between new infections or deaths at time t and the infectious people at time t - d where d is the previous infection-to-confirmation delays or the infection-to-death delays as appropriate.

The missing data (ie, confirmed cases or policy intervention) in the middle of time series were linearly interpolated using nonmissing observations.

Correlation Analysis

We conducted Spearman rank correlation analysis between Start-Date variable of each intervention and cumulative infection numbers using paired data from countries, to test whether the delayed policy implementation was associated with more infected cases. We chose Spearman rank correlation because the Start-Date variable (also the Stringency and Duration variables) has a highly skewed distribution. The correlation coefficient ranging from −1 to 1 is a statistical measure of the strength of a monotonic relationship between 2 variables. We also draw boxplots of the outcome by tertiles of Start-Date to examine the monotonic relationship. Because countries hit by the epidemic later may implement interventions timelier in view of the outbreaks in other countries, we performed an additional partial correlation analysis that was adjusted for the absolute date of first case occurrence.

We investigated the Spearman rank correlations of cumulative infection numbers with Stringency and Duration of interventions, separately at the early and middle stages of the epidemic. We adopted an approach proposed in a previous article that divided the progression curve of COVID-19 into the early slow growth phase, the middle fast growth phase, and the late steady phase, using a data-driven phenomenological logistic model. The slope of the curve represents the rate of epidemic growth, which is used to divide different phases. Examples of the fitted curves and phase cutoffs are shown in Appendix Fig. 2 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007. For sensitivity analysis, we alternatively defined the early phase as the first month since the first confirmed case occurrence for each country.

Counterfactual Effect Estimates

First, we compared the trends in the R before and after the implementation of each intervention for descriptive purpose. We calculated the mean value (95% confidence intervals [CIs]) of the R for all countries on different days relative to intervention implementation. As a summarization, we defined the commence date of any intervention as the median date of different intervention initiation dates if available and thus generated a new variable named “any intervention.” This simple “averaging method” provided a direct way for us to inspect the effect of interventions on R .

We introduced a new counterfactual estimator to infer causal relationships between interventions and R using time-series cross-sectional data (see Appendix Fig. 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007). Counterfactual estimator compares the observed outcomes with those one would expect if the intervention had not been implemented. In brief, the counterfactual estimator first constructs a model using time-series observations in the preintervention period and then takes observations under intervention as missing data and directly estimates their counterfactuals. This method has been detailed in previous articles and shown to provide more reliable causal effects than the conventional linear 2-way fixed effect models when the intervention effect is heterogeneous among units or there exist unobserved time-varying confounders. Therefore, the improved model was able to take account of the influences of time factor and country (unit) factor (such as population and Gross Domestic Product) on outcomes.

The original article provided 3 sets of counterfactual estimators and we chose the improved interactive 2-way fixed-effects model because of its ability to deal with time-varying confounders, as suggested by the author.

The model is as follows:

For any i = 1, 2, …, N and t = 1, 2, …, T,where Y is the outcome (R ) for country i at time t; D is intervention indicator that equals 1 if country i is under intervention at time t and equals 0 otherwise; δ it is the intervention effect on country i at time t; X is a (p×1) vector of exogenous covariates; β is a (p×1) vector of unknown parameters; f is an (r × 1) vector of unobserved common factors; and λ is an (r × 1) vector of unknown factor loadings. Intuitively, factors can be understood as time-varying trends that affect each country differently, and factor loadings capture their heterogeneous impacts caused by each country’s various unobserved characteristics. Here, the interactive component implicitly captures the effects of unobserved time-varying confounders and the effects of other policies, through which the influence of other policies is eliminated or controlled when performing estimations of δ . α and ξ are additive country and time fixed effects, respectively, and ε represents unobserved idiosyncratic shocks for country i at time t and has 0 mean.

The primary causal quantity of interest is the average intervention effect, which is an approximation of the estimated effects of the intervention on the outcome after policy implementation over time. Details about the calculation of average intervention effect were shown in Appendix Method 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007.

Results

Overall, time-series data of the intervention policies were retrieved for 178 countries on July 1, 2020, from the Oxford COVID-19 Government Response Tracker. After exclusion, a total of 145 countries were included in the current study. The number of countries in different continents is as follows: 36 in Europe, 36 in Asia, 47 in Africa, 13 in North America, 11 in South America, and 2 in Oceania. Cumulative infections per million population on July 1 ranges from 46 (Burundi) to 212 154 (Peru), with a median value of 9867 (interquartile range 2655-30 581).

Association of Intervention Start-Date With Cumulative Infections Per Million Population

Correlations between the Start-Date of 8 interventions and cumulative infections per million population are shown in Table 1 . Start-Dates of all 8 interventions were significantly and positively associated with the outcome, suggesting that the later intervention was commenced, the more infected cases that would be expected in that country. Start-Dates of public events cancellation (correlation coefficient [r] = 0.45), school closing (r = 0.43), and international travel controls (r = 0.43) showed the most pronounced associations. We displayed the boxplots of the outcome by tertiles of Start-Date in Figure 1 .

Table 1

Correlations of the Start-Date for 8 interventions with cumulative infections per million population on July 1, 2020.

Interventions	Correlation coefficients	95% CI
School closing	0.43	0.28-0.55
Workplace closing	0.28	0.12-0.42
Public events cancellation	0.45	0.31-0.57
Restrictions on gatherings	0.32	0.17-0.46
Public transport closing	0.22	0.06-0.37
Stay at home requirements	0.27	0.12-0.42
Restrictions on internal movement	0.27	0.12-0.42
International travel controls	0.43	0.29-0.55

Note. The correlation coefficients were calculated using Spearman rank correlation analysis. Start-Date for each intervention was the days of intervention initiation relative to the date of first cumulative 100 cases occurrence.

CI indicates confidence interval.

Figure 1

Association between Start-Date for 8 interventions and cumulative infections per million population on July 1, 2020, using boxplots. The figure shows boxplots of cumulative infections per million population by tertiles of Start-Date for each intervention using paired data from 145 countries. The boxes show the quartiles of the cumulative infections per million population. The whiskers extend to show the rest of the distribution, and the points are outliers.

The distributions of cumulative infections by tertitles of Start-Date for 8 interventions are presented in Figure 1. A similar monotonic increasing trend of cumulative infections along with Start-Date tertiles was observed for all interventions. Countries in tertile 3 demonstrated notably more infections and wider distributions than those in tertiles 1 and 2.

Additional partial correlation analysis that was adjusted for the absolute date of first case occurrence did not change the results substantially (see Appendix Table 1 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007). Sensitivity analysis by changing the definition of Start-Date variable to days relative to the first 10 cases occurrence showed that the positive associations still persist in most cases, although some are not significant (see Appendix Table 2 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007).

Association of Intervention Stringency and Duration With Cumulative Infections Per Million Population

The associations of intervention Stringency and Duration with cumulative infections per million population at different epidemic phases are presented in Table 2 and Appendix Table 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007. Most of the Stringency and Duration variables for the 8 interventions in the early phase (slow growth period) were negatively correlated with the outcome, with some of them showing significance (Table 2). Nevertheless, during the middle phase (the fast growth phase), the Stringency and Duration were mostly positively correlated with the outcome. The average duration of the early phase for all countries was 61 days. We conducted a further analysis by calculating Stringency and Duration in the first month and the second month of the epidemic, respectively. Similar results were found that the Stringency and Duration variables in the first month were mostly negatively, whereas in the second month were mostly positively, associated with cumulative infections (see Appendix Table 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007).

Table 2

Correlations of the Stringency and Duration for 8 interventions with cumulative infections per million population on July 1, 2020.

Interventions	Stringency		Duration
	Correlation coefficient	95% CI	Correlation coefficient	95% CI
Early phase
School closing	−0.14	−0.30 to 0.02	−0.15	−0.31 to 0.01
Workplace closing	−0.05	−0.11 to 0.21	−0.02	−0.18 to 0.15
Public events cancellation	−0.16	−0.31 to 0.00	−0.18	−0.33 to −0.02
Restrictions on gatherings	−0.02	−0.18 to 0.14	−0.10	−0.26 to 0.06
Public transport closing	0.04	−0.13 to 0.20	0.04	−0.13 to 0.20
Stay at home requirements	−0.05	−0.12 to 0.21	−0.01	−0.18 to 0.15
Restrictions on internal movement	−0.03	−0.14 to 0.19	−0.02	−0.14 to 0.18
International travel controls	−0.18	−0.33 to −0.02	−0.23	−0.38 to −0.07
Middle phase
School closing	0.11	−0.07 to 0.27	0.17	0.00-0.34
Workplace closing	0.34	0.17-0.48	0.34	0.18-0.48
Public events cancellation	0.15	−0.02 to 0.31	0.15	−0.02 to 0.32
Restrictions on gatherings	0.34	0.18-0.48	0.22	0.05-0.38
Public transport closing	0.15	−0.02 to 0.32	0.25	0.08-0.41
Stay at home requirements	0.18	0.01-0.34	0.24	0.07-0.39
Restrictions on internal movement	0.26	0.10-0.42	0.33	0.17-0.47
International travel controls	−0.03	−0.20 to 0.15	0.03	−0.15 to 0.20

Note. The correlation coefficients were calculated using Spearman correlation analysis, separately in the early phase and middle phase.

CI indicates confidence interval.

The results suggested some evidence that the longer and stricter implementations of some interventions in the very early phase but not the middle phase were associated with reductions in infected cases at the end.

Rt Before and After the Interventions

The COVID-19 R before and after the implementation of 8 interventions is presented in Figure 2 . Overall, a consistent similar pattern was observed for all interventions that R decreased slowly before the intervention, yet after the intervention, R decreased rapidly in 7 to 14 days, and the decreasing trend attenuated afterward. R converged to around 1 in approximately 30 days after the intervention. Overall, the average R decreased by 6.7% (95% CI 4.8-12.4) at 7 days and by 17.0% (95% CI 7.8-29.1) at 14 days after any of the interventions (see Appendix Fig. 4 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007).

Figure 2

COVID-19 R before and after the implementation of 8 interventions. The black lines show the mean value of R on different days relative to the start of intervention implementation. The shaded areas show corresponding 95% confidence intervals of R. The horizontal green lines indicate R equal to 1, and vertical green lines indicate the start of intervention implementation.

R indicates time-varying reproduction number.

Counterfactual Estimates for the Effects of Interventions on Rt

With counterfactual estimators, the average effects of different interventions by the time are presented in Figure 3 , and the average values for all periods after intervention are presented in Table 3 . All interventions give average estimates significantly < 0, among which the estimate of international travel controls is marginally significant. The test for no pretrend results is shown in Appendix Figure 5 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007. All 8 interventions have passed the equivalence test, suggesting the model successfully controlled for the effects of time-varying confounders and other interventions. In most cases, the average effect estimates surround zero in the preintervention period and decrease rapidly to below zero in 7 to 14 days after the intervention, to its minimum values (ranging from −0.52 to −0.08 with a median of −0.30) in 25 to 32 days (Fig. 3). This corresponds to a maximum 22% to 41% reduction in R . Among the 8 interventions, school closing, workplace closing, and public events cancellation demonstrated the strongest and most consistent evidence of associations.

Figure 3

Counterfactual estimates for the effects of 8 interventions on R. The curves and surrounding shaded areas show the average intervention effect estimates (with corresponding 95% confidence intervals) on R by time. The bar plots at the bottom indicate the number of countries under the related policy for each time period. The horizontal axes show the days relative to intervention, and numbers < 0 indicate the preintervention periods.

R indicates time-varying reproduction number.

Table 3

Counterfactual estimates for the average effects of 8 interventions on R.

Intervention	AIE on Rt	95% CI	P value
School closing	−0.29	−0.40 to −0.19	5.28E-08
Workplace closing	−0.29	−0.38 to −0.20	5.28E-11
Public events cancellation	−0.39	−0.52 to −0.27	7.12E-10
Restrictions on gatherings	−0.24	−0.35 to −0.14	7.63E-06
Public transport closing	−0.11	−0.20 to −0.03	3.55E-03
Stay at home requirements	−0.17	−0.25 to −0.08	1.02E-04
Restrictions on internal movement	−0.21	−0.28 to −0.14	7.58E-09
International travel controls	−0.20	−0.39 to −0.02	3.16E-02

Note. The data show the average effects of intervention policies on R for all countries and across postintervention periods.

AIE indicates average intervention effect; CI, confidence interval.

Robustness Analyses by Altering the Endpoint Date

Appendix Tables 4 to 7 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007 provide the robustness analyses for correlation results between variables of interventions and cumulative infections, and Appendix Tables 8 and 9 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.10.007 provide the robustness analyses for counterfactual effect estimates on R , by altering the cutoff date 1 month earlier (June 1, 2020) or later (August 1, 2020). The results demonstrate that the abovementioned findings are roughly unchanged, although the estimates differ to some extent.

Discussion

In this study, we found some evidence that earlier implementation, longer durations, and more strictness of containment policies at the early stage but not middle stage were associated with reduced infections of COVID-19. With a novel counterfactual estimator, we were able to control for the unobserved time-varying confounders, generating more reliable causal relationships. Our results showed that the government intervention policies were associated with a 22% to 41% reduction in COVID-19 transmission in approximately 25 to 32 days after their implementations.

Comparison With Previous Studies

The findings from our work align with those from previous studies,8, 13 , , 24, 25, 26, 27, 28 except that previous results mostly depended on modeling assumptions under presumptive scenarios or used data within a single country. Only a few studies assessed the impact of intervention policies for different countries using comparative methods. , , Nevertheless, these studies mainly depended on straightforward statistical methods, simply relating intervention policies to COVID-19 growth rate or R directly, which failed to account for time-varying confounders that affected the effect estimates. One study comparing the COVID-19 curve trends before and after interventions using data from 54 countries suggested that stay at home orders, curfews, and lockdowns curbed the increase in daily new case to < 5% within a month. Another study including 149 countries leveraged a simple meta-analysis method and synthesized the incidence rate ratios of COVID-19 before and after the implementation of physical distancing, concluding that physical distancing was associated with a 13% reduction in COVID-19 incidence. This study had less focus on the timeliness, strictness, and durations of interventions and was thus not able to conclude causal relationships. To the best of our knowledge, our study is the first study that addressed the issue of confounding using a novel counterfactual estimator based on an interactive 2-way fixed-effects model. The results from our study provided more reliable evidence and could better assist policy making.

Interpretation of Our Findings

We found that the early implementation of all containment policies was associated with reduced infection cases. This finding was as expected and in concert with most previous studies. , 29, 30, 31 Alongside this finding, we also found some evidence that the higher stringency and longer duration of some containment policies at the early or slow growth stage were correlated with reduced infection cases. Nevertheless, results from the middle or fast growth stage suggested evidence of positive associations. This is a novel finding that previous studies did not address. The positive associations between Stringency and Duration of intervention in the middle stage and total infections were probably attributed to reverse causality, which means that some countries strengthened and prolonged the interventions in face of more severe situations.

Recently, the new variants of severe acute respiratory syndrome coronavirus 2, especially the Delta variant that is more contagious than the original strain, spread rapidly in some countries such as India and the United States. Evidence regarding whether government intervention policies are effective on containing the new variants is scarce because most countries have loosened restrictions on public activities. Nevertheless, a recent regional outbreak of the Delta variant attacked Guangzhou and Shenzhen in China from May 2021 to June 2021. The local governments immediately enforced strict control measures since the identification of the first new case, including public events cancellation, unnecessary workplace closing, and contact tracing. The regional outbreak was successfully controlled within a month, preventing virus spillover and large-scale spreading. On the contrary, countries that did not implement strict containment measures at the very beginning are experiencing an uncontrollable domestic outbreak. This provides us some preliminary evidence that the early and stringent interventions are effective in controlling the outbreak of new variants. Given the strong transmissibility of the new variants, governments should enforce aggressive control measures as early as possible, even though the growth rate might be very slow in their countries at the early period. It seems too late to remedy when arriving at the fast growth stage.

The descriptive results from Figure 2 show that R demonstrated a decreasing trend before the intervention, which suggested that apart from the 8 interventions, some other unobserved factors such as public self-protective measures also had an effect on transmission reduction. Nevertheless, the preintervention period decreasing trend disappeared in Figure 3 with our counterfactual estimator, suggesting that our methods had successfully eliminated the effects of unobserved confounding factors and generated less biased effect estimates. Notably, we observed the strongest and consistent effects for school closing, workplace closing, and public events cancellation. All 3 containment policies were mandatory policies and more likely to take effects because it is easier to close public facilities.

Quantitatively, we found most interventions took their effects on reducing R rapidly about 7 to 14 days after implementation. The effects were strengthened by time to a maximum effect of around 30% reduction for R in 25 to 32 days. The estimates were similar to a previous study, except that our results provided the effect trends over time.

Limitations

Our study does have several limitations. First, the coding of intervention variables from Oxford COVID-19 Government Response Tracker relied on government announcements. Nevertheless, announcements did not guarantee mandatory implementation and people adherence varied because of the cultural and legal system differences. Second, because of the relatively small sample size for the number of countries, not all correlation analyses are significant, especially for Stringency and Duration; hence, those findings need to be interpreted with caution. Third, in addition to the public containment and closure policies, other personal protection strategies including wearing masks, quarantine, and hand hygiene also played an important role in epidemic mitigation. Those strategies were not the focus of our current study and have been addressed in previous researches. , , 35, 36, 37 Moreover, some intervention policies were often introduced in close temporal sequences. It has been difficult to untangle the individual effects. Although our counterfactual approach proved to have been largely controlled for the effects of time-varying confounders and other policies, statistical models dealing with confounders might be not perfect. Hence, results of interventions that are temporally correlated need to be interpreted with caution. Fourth, a large proportion of confirmed cases and deaths has been recorded from nursing homes including both residents and care workers. Nevertheless, data regarding the fraction of cases and deaths emerging from nursing homes were not available for most countries. We were not able to investigate the effects of the general policy interventions on nursing home epidemics at this point.

Conclusions

Using epidemiological data from 145 countries, we found some evidence that earlier, stricter, and longer implementation of containment policies at the early stage was associated with a reduction in infected cases. Moreover, the novel counterfactual estimator proved to have generated more reliable intervention effect estimates of policies. Our results provided evidence of the quantitative effect of different policy intervention over time. Those findings shall have important implications for governments to enact or lift containment policies in fighting against the current and future waves of the COVID-19 outbreak. Future studies should emphasize on how adding and removing intervention policies affect the transmission of the virus, especially its new mutants such as the Delta variant, for decision making on lifting containment policies.

Article and Author Information

Author Contributions:Concept and design: Sun, Zheng, Liang, Yang, Li, Luo, He, Zhong

Acquisition of data: Sun, Luo

Analysis and interpretation of data: Sun, Zheng, Liang, Yang, Zeng, Li, Luo

Drafting of the manuscript: Sun, Zheng, Zeng, Luo

Critical revision of the paper for important intellectual content: Sun, Zheng, Liang, Yang, Zeng, Li, Luo, Alexander Ng, He

Statistical analysis: Sun, Luo

Administrative, technical, or logisticsupport: Alexander Ng, Zhong

Supervision: Alexander Ng, He, Zhong

Conflict of Interest Disclosures: The authors reported no conflicts of interest.

Funding/Support: This work was funded by Key-Area Research and Development Program of Guangdong Province, China (No. 2018B010111001).