Lifting Statewide Mask Mandates and COVID-19 Cases: A Synthetic Control Study.

IMPORTANCE: As states reopened their economies state and local officials made decisions on policies and restrictions that had an impact on the evolution of the pandemic and the health of the citizens. Some states made the decision to lift mask mandates starting spring 2021. Data-driven methods help evaluate the appropriateness and consequences of such decisions.
OBJECTIVE: To investigate the association of lifting the mask mandate with changes in the cumulative coronavirus case rate.
DESIGN: Synthetic control study design on lifting mask mandate in the state of Iowa implemented on February 7, 2021.
SETTING: Daily state-level data from the COVID-19 Community Profile Report published by the US Department of Health & Human Services, COVIDcast dataset of the Delphi Research Group, and Google Community Mobility Reports. EXPOSURES AND OUTCOME: Mask mandate policy lift at the state level. State-day observations of the cumulative case rate measured as the cumulative number of new cases per 100,000 people in the previous 7 days.
RESULTS: The cumulative case rate in Iowa increased by 20%-30% within 3 weeks of lifting the mask mandate as compared with a synthetic control unit. This association appeared to be related to people, in fact, reducing their mask-wearing habits.
CONCLUSIONS: Lifting the mask mandate in Iowa was associated with an increase in new COVID-19 cases. Caution should be applied when making this type of policy decision before having achieved a more stable control of the pandemic.

Adopting or lifting mask mandates and other policies related to the control of the virus SARS-CoV-19 and COVID-19 disease has been a topic of debate since the beginning of the pandemic. Countries, states, counties, and individual municipalities were continually faced with the dilemmas of what policies to implement and what policies to lift to protect the health of their citizens.

Proponents and adversaries of mask mandates often based their arguments and decision-making on health official recommendations and data-driven methods. Yet, it is often the case that some of the policy decisions at the state or county level seem to defy and contradict the national recommendations. This debate has evolved during the several phases of the pandemic.

The core objective of these policies is often to determine a decrease in the number of new COVID-19 cases in the region in which they are implemented by way of affecting citizens’ behavior. Several authors have studied not only the effects of such policy implementations but also a vast array of counterfactuals to show what other implications could have resulted from implementing different policies.1,2 For example, early in the pandemic, the state government mandate for use of face masks in 15 states was estimated to have significantly caused a decline in the growth rate of daily COVID-19 cases in the period following the implementation of these policies.2 Similarly, the mask mandate issued in the state of New York was linked to a decrease in cases and deaths that was significantly greater than Massachusetts, which did not implement a mask mandate at the same time.3 Furthermore, mask mandates have been associated with a decline in weekly COVID-19–associated hospitalization growth rates.4 As states started to reopen during the summer, other authors looked at the excess COVID-19 cases and deaths after reopening.5 These studies utilized the natural experiment or quasi-experiment setting of policy implementation as a base for their causal inference. Yet, to date, there are a limited number of studies that investigated how different reopening strategies performed.6

In July 2020, the state of Kansas issued an executive order to require mask-wearing in public places. Counties were free to opt-out of the state mandate, and more than 75% of them did. A study of this mask mandate found a significant decrease of cases in the mandated counties, as opposed to the significant increase in the counties that opted out.7 The Institute of Health Metrics and Evaluation (IHME) COVID-19 Forecasting Team used a counterfactual modeling approach to model several alternative scenarios for the United States in the fall-winter 2020, and predicted that 130,000 lives could be saved with the most universal implementation of social distancing and mask use measures.8

With the new downward trend in the number of cases and the vaccination roll-out in early 2021, states started to lift mask mandates again or to discuss the possibility. Proponents of these policies argued that the current situation was completely different than the previous months and that the consequence of lifting mask mandates would not be as negative as before. The strongest argument in favor of lifting mask mandates was the ongoing vaccination roll-out and the endogenous downward trend in the pandemic curve of the number of cases, nationally and locally. Interestingly, the debate in early 2022 echoed many of the previous points.

To contribute to this debate, we examined the association that lifting the mask mandate in Iowa on February 7, 2021, had on the development of COVID-19 cases in this state. We used a synthetic control study design to model a simulated development of cases had the mask mandate not been lifted.

METHODS

Population, Measures, and Data Sources

We were interested in examining the effect of lifting mandates at state level. We focused on Iowa because it provided the clearest analysis of the 4 US states that lifted their mask mandate in February 2021. Mississippi and Texas lifted their mask mandate as well, but the winter storm in February affected the reporting of COVID-19 cases.9 Montana lifted the statewide mask mandate but explicitly left the final decision to the counties.

We collected data for the state of Iowa and the 31 US states that had a mask mandate in place and did not lift it during the study period—these states constitute the donor pool in the synthetic control method. This donor pool size compares favorably to other synthetic control studies which use 13,10 16,11 or 4210 units in the donor pool.

Overall, the dataset of the cumulative case rate included 1440 state-day observations between January 25, 2021, and March 10, 2021. The beginning of the study period was limited by the availability of vaccine data, which started in the community profile reports on January 15, 2021. The cumulative case rate was measured as the 10-day lag of confirmed cumulative COVID-19 cases in the previous 7 days per 100,000 people. The rationale of this choice was to proxy the time when a person is infected as being before the time it is discovered and confirmed as a positive case.

Several variables were used as predictors in the synthetic control method to form a weighted average of states in the control group, the synthetic control unit. These variables included: proportion of cumulative infections (total infection prevalence) measured as a 10-day lag of the percentage of a state’s population that was infected at any point in time; proportion of vaccines measured as a 10-day lag of the percentage of vaccines relative to a state’s population; 10-day lags of several mobility measures, detailed below; proportion of “mask wearing of others” measured as a 10-day lag of the percentage of people who say that most or all other people wear masks when they are in public; and states’ population density.

Data on COVID-19 cases and vaccinations were obtained from the COVID-19 Community Profile Report published by the US Department of Health & Human Services.12 These data were collected daily from the Data Strategy and Execution Workgroup in the Joint Coordination Cell, under the White House COVID-19 Team. Data on reported mask wearing, as measured by the proportion of people who said that most or all other people wore masks when they were in public and social distancing was not possible, were publicly available via the COVIDcast dataset on the Delphi Research Group website13; data from this source were used in prior studies.14–16 Mobility data were obtained from the Google Community Mobility Reports17; in particular we used data that showed the percentage change of visits at different categorized places compared to a baseline day, which is the median value from the 5‑week period January 3 to February 6, 2020. Google community reports show how visits at different places changed compared with a baseline. We use the 6 mobility categories of retail, residential, grocery, parks, workplace, and transit. The percentage represents the number of people compared with baseline. This relative percentage is often negative because people’s mobility in 2021 was highly reduced compared with 2020.

Synthetic Control Method

To estimate the association that lifting the mask mandate on February 7, 2021 had on the development of COVID-19 cases in Iowa, we utilized the synthetic control estimator.18,19 This estimator allowed us to construct a counterfactual, a synthetic unit, that simulated the development of COVID-19 cases in Iowa had the mask mandate never been lifted. We modeled the synthetic unit as a weighted average of the 31 US states that did not lift the mask mandate throughout the study period (January 25 to March 10).

Formally, we were interested in estimating , that is the difference between the COVID-19 cases in state s on calendar date t, if the mask mandate is lifted and if it is kept in place . Of these 2 outcomes, only 1 could be observed. With the synthetic control estimator, we could model the other, counterfactual, outcome of Iowa not lifting the mask mandate on February 7. The synthetic control method uses an optimally chosen linear combination of the 31 US states that did not lift the mask mandate as a synthetic control unit. The difference between real and synthetic unit could then be estimated as:

where were the cumulative case rate in Iowa, were the cumulative case rates in the 31 US states in the control group, and w were the optimally chosen weights for every US state in the control group. To create a synthetic control unit as similar as possible to Iowa, we estimated the weighting matrix W* by minimizing

where X 1 were the preintervention predictors for Iowa, X 0 were the preintervention predictors for the 31 US states in the control group, and V was a (k×k) symmetric and positive semidefinite matrix that weighted the prediction variables. The estimation of W* was subject to the constraints that and that . Following general practice, we chose V* so that the preintervention mean squared prediction error was minimized.

We assessed the significance of an effect with permutation-based P-values because the synthetic control method does not generate traditional P-values.10 It may be possible to argue that the difference between the synthetic unit and the actual state of Iowa was due to chance or a failure to accurately reproduce the counterfactual development of COVID-19 cases. Therefore, we repeated the analysis for the 31 US states that did not lift their mask mandate on February 7, or at any other point during the study period. We then calculated the post/pre mean squared prediction error (MSPE) ratio, that is the ratio of the root MSPE after and before the treatment date. The permutation-based P-values were the fraction of estimated post/pre MSPE ratios that are as large as the one estimated for Iowa. These permutation-based P-values should be treated as suggestive of an effect rather than for traditional null hypothesis-based inference.11

After we established the association between lifting the mask mandate and the development of COVID-19 cases, we matched the synthetic unit with the COVID-19 cases in each preintervention day. This approach reduces the weight of other covariates but has been used in similar research as an alternative model.20 We also conducted a range of robustness checks to support our analysis. First, we varied the lag for the predictors from 10- to 5-day lags. Second, we excluded mobility statistics from the predictors because these might have been influenced by the intervention itself of lifting the mask mandate. Third, we excluded vaccination data and extended the preintervention period.

Finally, we conducted a post hoc analysis to understand the mechanism better. If lifting the mask mandate was associated with an increase in COVID-19 cases, we expected to see a decline in mask wearing after the policy change. Therefore, we re-estimated the synthetic control estimator with the mask-wearing of others variable as the dependent variable. Lastly, we reran the analysis for the lifting of the mask mandate in Mississippi (on March 3) and Texas (on March 10).

All analyses were performed with R version 4.0.2 using the synth function from the Synth package.21

RESULTS

We established the association between lifting the mask mandate and COVID-19 cases, by comparing the difference in cumulative case rate between the real Iowa and the synthetic control unit 21 days after the mask mandate was lifted.

Table 1 displays the descriptive statistics for predictor variables used to construct the synthetic control Iowa. The sample means for both Iowa and Synthetic Iowa are displayed, and the sample means for the 31 US states in the control group. Lastly, the weights V* for the predictor variables are shown in the last column. Values for the treated and synthetic Iowa were consistent for most predictors. Notable exceptions were the population density, which was lower for Iowa than the synthetic Iowa. This might have biased the results toward 0, as a higher population density might be associated with higher cumulative case rate, but this predictor had a very small weight of 0.016. The values for other predictors with weights >0.01 were very similar for the treated (real) and synthetic Iowa.

TABLE 1

Descriptive Statistics (Means) for Predictor Variables Used to Construct Synthetic Iowa, January 25, 2021 to February 6, 2021

Variable	Iowa	Synthetic Iowa	Average for 31 Control States	Variable Weight (V*)
Proportion of cumulative infections to total population (total infection prevalence)	0.097	0.096	0.070	0.354
COVID-19 cumulative case rate (cases per 100,000 people in previous 7 d)	281	323	385	0.077
Percentage of vaccine doses to total population	9.84%	9.80%	10.62%	0.179
Retail mobility	−20.14%	−24.82%	−27.12%	0.039
Residential mobility	9.71%	9.62%	10.98%	0.067
Grocery mobility	−2.86%	−11.15%	−14.15%	0.012
Parks mobility	−12.79%	−1.47%	−12.91%	0
Workplace mobility	−20.29%	−21.16%	−28.46%	0.074
Transit mobility	−26.71%	−26.16%	−37.42%	0.095
Mask wearing of others	78.80%	79.03%	86.63%	0.087
Population density	56.716	113.342	632.946	0.017

The table presents the predictor variable mean values for the real Iowa, the synthetic control unit and the average of the 31 control states. All variables except population density were 10-day lagged values and reported as the mean over the preintervention period (January 25 to February 6). The variable weight (V*) is the predictor weight that was used to construct the synthetic Iowa based on 31 states in the donor pool. The donor pool states are Alabama, Arkansas, California, Colorado, Connecticut, District of Columbia, Illinois, Indiana, Kansas, Kentucky, Louisiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Nevada, New Hampshire, New Jersey, New Mexico, New York, North Carolina, Ohio, Oregon, Pennsylvania, Rhode Island, Utah, Virginia, Washington, West Virginia, and Wisconsin.

We constructed a synthetic Iowa as a weighted average of the 31 US states in the control group. The weights are chosen so that the resulting synthetic Iowa best reproduced the values of the predictors (Table 1) of COVID-19 cases in Iowa in the preintervention period. The resulting synthetic Iowa was a weighted average of Wisconsin (0.81), Indiana (0.114), and Arkansas (0.075). All other states in the control group obtained 0 weights. To alleviate any concern that our results for Iowa were due to an abnormal change in the cumulative case rate for Wisconsin we reran our estimator with Wisconsin as the treated unit. We found no strong evidence that Wisconsin experienced an abnormal change in the cumulative case rate over our study period.

Figure 1 displays the trends in cumulative case rate for Iowa and the synthetic Iowa. Between January 20 and February 7, the number of COVID-19 cases in the population was steadily declining. Starting at 248 cases on January 20, down to 171 on February 7. After the mask mandate was lifted on February 7, the cumulative case rate started to level off for Iowa, while they continued to decline for the synthetic unit so that a visible gap appeared within 1 week after lifting the mask mandate. Table 2 provides the results of our analysis and robustness checks. The estimated absolute change in cumulative case rate associated with lifting the mask mandate, was 28.2 (corresponding to a 30.7% above synthetic unit level) 21 days after lifting the mask mandate.

FIGURE 1

The vertical line indicates the day (February 7) the mask mandate was lifted. Synthetic Iowa is a weighted average of control states that best approximated the trend before lifting the mask mandate. These results were suggestive of an effect with a permutation-based P-value of 3/32 or 0.094. The analytic study sample included daily data from 32 US states for COVID-19 case rates and a set of predictor variables. The observation window was January 25, 2021 to March 10, 2021. Refer to Table 1 for summary statistics.

TABLE 2

Association Between Lifting the Mask Mandate and Changes in Outcome Variables in Iowa

	Days After Lifting Mask Mandate
Results	0 d	21 d
Main analysis
Iowa’s COVID-19 cumulative case rate	171	120
Synthetic control’s COVID-19 cumulative case rate	177.0	91.8
Estimated absolute [relative] change associated with lifting the mask mandate		28.2 [30.7%]
Permutation-based P-value	0.094 [3/32]
Alternative model: matching on preintervention outcomes
Iowa’s COVID-19 cumulative case rate	171	120
Synthetic control’s COVID-19 cumulative case rate	177	91.9
Estimated absolute [relative] change associated with lifting the mask mandate		28.1 [30.6%]
Permutation-based P-value	0.094 [3/32]
Robustness check I: 5-day lag
Iowa’s COVID-19 cumulative case rate	171	120
Synthetic control’s COVID-19 cumulative case rate	175.5	93.0
Estimated absolute [relative] change associated with lifting the mask mandate		27.0 [29.1%]
Permutation-based P-value	0.094 [3/32]
Robustness check II: without mobility predictors
Iowa’s COVID-19 cumulative case rate	171	120
Synthetic control’s COVID-19 cumulative case rate	179.3	93.1
Estimated absolute [relative] change associated with lifting the mask mandate		26.9 [28.8%]
Permutation-based P-value	0.094 [3/32]
Robustness check III: without vaccination predictor
Iowa’s COVID-19 cumulative case rate	171	120
Synthetic control’s COVID-19 cumulative case rate	167.1	91.6
Estimated absolute [relative] change associated with lifting the mask mandate		28.4 [31.0%]
Permutation-based P-value	0.531 [17/32]
Post hoc analysis
Iowa’s reported percentage of others wearing masks	77.2	71.7
Synthetic control’s reported percentage of others wearing masks	78.9	76.9
Estimated absolute [relative] change associated with lifting the mask mandate		−5.2 [−6.8%]
Permutation-based P-value	0.063 [2/32]

The estimated absolute [relative] change associated with lifting the mask mandate is calculated as the mean difference between Iowa and the synthetic Iowa. Permutation-based P-values from are calculated as rank of Iowa’s post/pre MSPE ratio divided by total number of donor pool states [Rank Iowa/Number of donor pool states].

The results of the placebo tests had Iowa with the third highest post/pre MSPE ratio of all states in our sample (post/pre MSPE ratio: 33.2). Only Nevada (70.7) and California (38.2), which both experienced a higher-than-expected decline in cases in mid-February, had a higher ratio. This was suggestive of an effect with a permutation-based P-value of 0.094 and meant that the probability to obtain an estimated change at least as great as Iowa was 9.4% if the intervention was reassigned at random to the other states.

Our findings were robust to an alternative specification and a range of sensitivity checks (Table 2). First, we matched the synthetic unit with the COVID-19 cases in each preintervention day.22 This alternative specification reduced the weights of the other predictors considerably with the preintervention outcomes taking up 78.7% of the weight. The estimated absolute change in cumulative case rate associated with lifting the mask mandate, remains 28.1 (30.6%) 21 days after lifting the mask mandate. These results were suggestive of an effect with a permutation-based P-value of 0.094. When using a 5-day lag instead of a 10-day lag as a robustness check, the estimated absolute change in cumulative case rate associated with lifting the mask mandate was 27.0 (29.1%). Iowa had the third highest post/pre MSPE ratio of all states in our sample (permutation-based P-value: 0.094). Next, when dropping the mobility predictors, the estimated change after lifting the mask mandate was 26.9 (28.8%). Iowa had the third highest pre/post MSPE ratio of all states in our sample (permutation-based P-value: 0.094). Lastly, we dropped vaccination as a predictor and extended the preintervention period to January 16. For this specification, the preintervention fit between Iowa and the synthetic Iowa was less accurate than before. This led to a lower post/pre MSPE ratio and permutation-based P-value. However, the estimated change after lifting the mask mandate remained positive 28.4 (31.0%).

The post hoc analysis showed that Iowa experienced a decline in the reported percentage of people who said that most or all other people wore masks when they were in public and social distancing was not possible (Figure 2). The percentage declined from 79.0% on January 25 to 69.8% on March 8. The percentage for the synthetic Iowa declined less steeply from 78.9% to 76.9% over the same period. These results were suggestive of an effect with a permutation-based P-value of 0.063 (Table 2).

FIGURE 2

The vertical line indicates the day (February 7) the mask mandate was lifted. Synthetic Iowa is a weighted average of control states that best approximated the trend of others who were reported to wear a mask before lifting the mask mandate. These results were suggestive of an effect with a permutation-based P-value of 2/32 or 0.063. The analytic study sample included daily data from 32 US states for reported mask-wearing of others and a set of predictor variables. The observation window was January 25, 2021 to March 10, 2021. Refer to Table 2 for detailed results and inference.

DISCUSSION

In this study, we examined the association that lifting the statewide mask mandate in Iowa had on the development of cumulative case rates measured as cumulative number of cases per 100,000 people in the previous 7 days. While past research pointed to the benefits of masks,23 there were also doubts about the usefulness of mask mandates in the spring of 2021.20,24

State officials believed that people would wear masks even without a mandate and that vaccinations would help to keep cases in check. We found that lifting the mask mandate in Iowa was linked to an increase in cases in comparison with the development of cases in a synthetic control unit. The results were suggestive of an effect with a permutation-based P-value of 0.094. Furthermore, we found that Iowans had a more rapid decline in mask wearing than their synthetic counterpart. Taken together, these results suggested that lifting the mask mandate was linked to a decline in abidance to mask recommendations and an increase in COVID-19 cases.

These findings augment the existing research on the benefits of mask-wearing1,23,24 and reopening policy6,25,26 by emphasizing the COVID-19 trends associated with lifting mask mandates in Iowa in spring 2021. The results were in contrast to studies of Texas where lifting the mask mandate and reopening did not seem to be associated with an increase in mobility or infections.20 Our study took into account the mask wearing behavior of individuals and Iowa did not suffer from reporting discontinuities due to a winter storm.9

Limitations

The study had several limitations that might have influenced the results. While we included many covariates that should not be affected by lifting the mask mandate, there may be unobserved predictors of COVID-19 cases that would have influenced the findings. This concern is reduced, because the synthetic control method does balance on unobserved heterogeneity, given a long enough preintervention period.27

Also, it would have been informative to examine other COVID-19–related outcomes such as hospitalization and deaths. These measures had a higher standard deviation of the time between infection and measured outcome28 which made estimating effects based on daily developments infeasible. While Iowa lifted the statewide mask mandate, certain cities introduced their own mandates and many businesses still required masks. Therefore, our results were biased towards zero and would likely have been higher, had all cities and establishments complied.

CONCLUSIONS

In this synthetic control study design leveraging Iowa’s lifting of mask mandate, we found that lifting mask mandates was associated with an increase in COVID-19 cases. These results were consistent with findings on the association of introducing mask mandates in Kansas8 and reopening decisions in the United States29 and across Europe.6 Our findings added a more nuanced understanding of the mechanism by showing that while mask-wearing was reduced, mobility of people did not increase. The results suggested that containment policies such as mask mandates should be kept in place at least until population immunity through widespread vaccinations is achieved. However, further work is needed on examining whether mask mandates can be lifted once more individuals are vaccinated as the vaccination rate in our data was too low to study this conclusively.