The effect of state-level stay-at-home orders on COVID-19 infection rates.

State-level stay-at-home orders were monitored to determine their effect on the rate of confirmed COVID-19 diagnoses. Confirmed cases were tracked before and after state-level stay-at-home orders were put in place. Linear regression techniques were used to determine slopes for log case count data, and meta analyses were conducted to combine data across states. The results were remarkably consistent across states and support the usefulness of stay-at-home orders in reducing COVID-19 infection rates.

Background

The emergence of the COVID-19 virus and the lack of therapeutic responses have led to widespread implementation of nonpharmaceutical interventions (NPIs) across the United States. Substantial evidence exists for the effectiveness of NPIs in preventing the spread of infection. , However, significant controversy still exists about the value and cost-benefit of these interventions in response to the current pandemic. Specifically, it is unclear what social isolation actions must be mandated by government policy to reach desired mitigation outcomes.

In response to the rising death toll associated with COVID-19, 42 states and the District of Columbia have implemented some variation of a state-wide stay-at-home order. In each case, there was controversy regarding the appropriateness of the order, and it is widely understood that these decisions incur human and economic costs. In the current analysis, we present data from the states with stay-at-home orders and examine the effect of these policies on the rate of increase in COVID-19 diagnoses.

Methods

Data sources

Google searches were conducted daily to assess state-level responses to the COVID-19 pandemic. Once stay-at-home orders were sighted, state government webpages were searched to confirm the dates that the orders went into effect. Forty-two states and the District of Columbia were identified to have issued a stay-at-home order between March 19, 2020 and April 7, 2020. Stay-at-home orders were defined as statewide mandates that included (a) mandatory nonessential business closures, (b) furloughs enforced for most government and commercial employees, (c) prohibition of public events and gatherings, and (d) travel restrictions including orders to avoid leaving the home except for necessities such as groceries and medical care. Daily confirmed COVID-19 cases were obtained from the Johns Hopkins Center for Health Security Application and downloaded via GitHub.

Analysis

COVID-19 confirmed case counts were matched to stay-at-home order dates and divided into preorder and postorder datasets for every state. Eight states without state-wide stay-at-home orders were excluded from this analysis. For the pre- and postorder sets, slopes were calculated from both raw and logged case count data using linear regression techniques and R2 fit statistics were obtained. As expected, R2 fits were substantially better for the logged data and this approach was used throughout the analysis. Ninety-five percent confidence intervals were obtained for all parameters. Meta-analytic techniques were used to combine data across states using the METAN command suite in Stata. A sensitivity analysis was conducted using the same linear regression and meta-analytic approach to determine whether delaying the days counted as postorder by 1 week would impact the infection rates and R2 fit statistics.

Results

Table 1 displays the dates stay-at-home orders went into effect as well as logged, preorder and postorder slopes and 95% confidence intervals for all 42 states and the District of Columbia. R2 fit statistics are also shown. In general, the fit of the slopes to the logged data is excellent, as would be expected from an exponentially growing pandemic in a naive population. The average rate of increase preorder was 0.113 (95% C.I.: 0.110, 0.115) per day and postorder was 0.047 (95% C.I.: 0.045, 0.048) per day

Table 1

Infection rates before and after stay at home orders went into effect*,†

State	Order date	Number of days before order	Infection rate and confidence interval (before order)	R squared (before order)	Number of days after order	Infection rate and confidence interval (after order)	R squared (after order)
Alabama	4/4/2020	21	0.099 (0.088, 0.109)	0.951	9	0.042 (0.039, 0.045)	0.994
Alaska	3/28/2020	9	0.11 (0.095, 0.126)	0.976	16	0.03 (0.027, 0.032)	0.982
Arizona	3/31/2020	18	0.134 (0.124, 0.143)	0.981	13	0.03 (0.025, 0.036)	0.932
California	3/19/2020	28	0.084 (0.077, 0.091)	0.957	25	0.055 (0.05, 0.06)	0.959
Colorado	3/26/2020	17	0.11 (0.1, 0.121)	0.973	18	0.04 (0.035, 0.044)	0.951
Connecticut	3/23/2020	11	0.154 (0.136, 0.172)	0.976	21	0.065 (0.059, 0.07)	0.968
Delaware	3/24/2020	8	0.111 (0.096, 0.126)	0.983	20	0.064 (0.061, 0.066)	0.993
District of Columbia	4/1/2020	17	0.091 (0.082, 0.099)	0.972	12	0.044 (0.038, 0.049)	0.974
Florida	4/3/2020	27	0.12 (0.114, 0.127)	0.981	10	0.029 (0.028, 0.03)	0.997
Georgia	4/3/2020	26	0.109 (0.102, 0.116)	0.979	10	0.039 (0.032, 0.045)	0.959
Hawaii	3/25/2020	9	0.117 (0.095, 0.14)	0.956	19	0.04 (0.034, 0.046)	0.92
Idaho	3/25/2020	7	0.097 (0.072, 0.122)	0.951	19	0.054 (0.043, 0.064)	0.874
Illinois	3/21/2020	12	0.158 (0.142, 0.173)	0.981	23	0.059 (0.054, 0.064)	0.965
Indiana	3/24/2020	14	0.123 (0.108, 0.138)	0.964	20	0.061 (0.054, 0.069)	0.94
Kansas	3/30/2020	15	0.105 (0.098, 0.112)	0.987	14	0.039 (0.036, 0.043)	0.982
Kentucky	3/26/2020	15	0.102 (0.095, 0.109)	0.987	18	0.046 (0.042, 0.049)	0.98
Louisiana	3/23/2020	12	0.159 (0.141, 0.177)	0.976	21	0.061 (0.054, 0.068)	0.945
Maine	4/2/2020	18	0.072 (0.065, 0.08)	0.961	11	0.019 (0.018, 0.021)	0.991
Maryland	3/30/2020	19	0.115 (0.109, 0.12)	0.991	14	0.057 (0.053, 0.06)	0.991
Massachusetts	3/24/2020	17	0.098 (0.086, 0.109)	0.958	20	0.057 (0.052, 0.063)	0.965
Michigan	3/24/2020	12	0.197 (0.183, 0.211)	0.99	20	0.054 (0.047, 0.061)	0.938
Minnesota	3/27/2020	15	0.102 (0.09, 0.114)	0.962	17	0.034 (0.033, 0.036)	0.992
Mississippi	4/3/2020	19	0.107 (0.091, 0.123)	0.919	10	0.034 (0.032, 0.037)	0.992
Missouri	4/6/2020	21	0.113 (0.099, 0.128)	0.934	7	0.029 (0.023, 0.034)	0.971
Montana	3/28/2020	11	0.113 (0.101, 0.125)	0.982	16	0.027 (0.023, 0.03)	0.956
Nevada	4/1/2020	21	0.104 (0.097, 0.111)	0.979	12	0.029 (0.026, 0.032)	0.982
New Hampshire	3/27/2020	12	0.081 (0.07, 0.092)	0.964	17	0.042 (0.036, 0.049)	0.924
New Jersey	3/21/2020	12	0.182 (0.169, 0.196)	0.989	23	0.066 (0.058, 0.074)	0.927
New Mexico	3/24/2020	12	0.09 (0.084, 0.096)	0.99	20	0.058 (0.053, 0.064)	0.966
New York	3/22/2020	19	0.164 (0.153, 0.174)	0.985	22	0.045 (0.04, 0.049)	0.955
North Carolina	3/30/2020	19	0.115 (0.107, 0.124)	0.98	14	0.038 (0.035, 0.042)	0.982
Ohio	3/23/2020	11	0.15 (0.138, 0.161)	0.99	21	0.053 (0.047, 0.059)	0.948
Oregon	3/23/2020	16	0.082 (0.076, 0.087)	0.985	21	0.04 (0.034, 0.045)	0.93
Pennsylvania	4/1/2020	23	0.126 (0.122, 0.131)	0.993	12	0.047 (0.042, 0.053)	0.975
Rhode Island	3/28/2020	16	0.086 (0.08, 0.091)	0.988	16	0.062 (0.058, 0.065)	0.991
South Carolina	4/7/2020	28	0.091 (0.082, 0.099)	0.948	6	0.028 (0.014, 0.042)	0.887
Tennessee	4/2/2020	22	0.112 (0.104, 0.121)	0.973	11	0.026 (0.024, 0.028)	0.99
Texas	4/2/2020	26	0.114 (0.108, 0.12)	0.984	11	0.041 (0.037, 0.045)	0.985
Vermont	3/25/2020	10	0.125 (0.109, 0.141)	0.975	19	0.038 (0.035, 0.042)	0.968
Virginia	3/30/2020	19	0.099 (0.094, 0.105)	0.988	14	0.051 (0.047, 0.054)	0.986
Washington	3/23/2020	23	0.101 (0.093, 0.11)	0.966	21	0.033 (0.029, 0.036)	0.951
West Virginia	3/24/2020	4	0.079 (-0.002, 0.16)	0.899	20	0.061 (0.054, 0.067)	0.951
Wisconsin	3/25/2020	13	0.133 (0.12, 0.147)	0.977	19	0.036 (0.032, 0.039)	0.964

* The COVID-19 infection rates above are logged, preorder and postorder slopes.

† Results shown above were updated through April 13, 2020.

Fig 1 presents meta analyses results using combined data across states. Two meta analyses were conducted using different sets of weights: number of days during the period and final number of cases during the period. Weighing by number of days yielded a pooled standardized mean difference of 3.847 (95% C.I.: 3.653, 4.041; P < .0001). Weighing by final number of cases yielded a similar result, with a pooled standardized mean difference of 6.847 (95% C.I.: 6.831, 6.863; P < .0001). Overall, these slope changes translate to a reduction from about 12% more cases per day (and thus a 5 to 6-day doubling rate) to 5% more cases per day (and thus a 14-day doubling rate).

Fig 1

(a) Meta-analysis weighed by the number of days. 95% confidence intervals (CI) and standardized mean difference (SMD) are shown. (b) Meta-analysis weighed by the final number of cases 95% confidence intervals (CI) and standardized mean difference (SMD) are shown.

The same analyses were repeated with postorder datasets starting 1 week after stay-at-home orders went into effect. In this sensitivity analysis, the pooled standardized mean difference was 4.229 (95% C.I.: 4.011, 4.448; P < .0001) after weighing by number of days and was 6.811 (95% C.I.: 6.798, 6.824; P < .0001) after weighing by final number of cases.

Discussion

These data suggest a remarkably consistent and important effect associated with the issuing of stay-at-home orders and are generally supportive of such measures. In combination with other social isolation approaches and NPIs, these orders may play a significant role in “flattening the curve.” However, this study has a number of important limitations. First, it would be impossible to isolate the effect of these orders against the background of numerous other local, state, and federal interventions occurring at the same time. Second, the expected COVID-19 expansion curve in the absence of interventions to reduce the transmission is unknown. As such, it is possible that the observed slope changes are driven by the natural history of the pandemic as well as the specific policies in question. Two sensitivity analyses were conducted to examine whether the number of days in the pre- and postperiods were drivers of either the slopes or the change in the slopes between periods. In both cases, we found no relationship. Third, states implemented stay-at-home orders in response to the pandemic and these observations are thus profoundly threatened by selection and indication biases. Fourth, even with the use of trend data in this analysis, there is a threat of regression to the mean if stay-at-home orders were consistently placed at the peak of epidemic growth.

Finally, and perhaps most importantly, a major limitation of this work is the endogenous relationship between case counts and both the availability and use of testing. The availability of testing was a significant barrier to COVID-19 diagnoses in the early stages of the pandemic in the United States and improved gradually over the month of March. Tests per day increased roughly 10-fold between March 15th and March 31st and have increased at a more gradual rate of 4-fold over the last 6 weeks. We conducted an additional sensitivity analysis to examine whether the decrease in case incidence between pre- and postorder periods was different if the order occurred before or after April 1st and found the change in slopes to be nearly identical. As mentioned earlier, we also conducted a sensitivity analysis to account for the effect of time on these slopes and found no effect. Overall, the results must be interpreted in the context of this threat. Not only was the availability of tests evolving over this time period, but there is a direct relationship between the number of infections and public awareness of the pandemic and the number of tests performed. As such, it would be difficult to begin to parse out the effect of testing on the rate of incidence of COVID-19. That said, it is likely that the availability of tests may have an unmeasured impact on these results, but we cannot know if the impact was to increase or decrease the incidence rates. Despite these limitations, the consistency and strength of the results is notable. The results presented here will be updated daily and available for public download at www.hpmcovidpolicy.org.

Conclusions

Stay-at-home orders are generally supported by our results. This data is publicly available and provides daily updates on national infection rates. By tracking stay-at-home orders and the resulting infection rates, information is gathered on the effectiveness of such orders in mitigating the spread of COVID-19. The data may also inform policy and mitigation efforts for future infection outbreaks. Further research is needed to examine the effect of these policies in the longer term and in combination with other interventions.