Public health impact of covid-19 vaccines in the US: observational study.

OBJECTIVE: To evaluate the impact of vaccine scale-up on population level covid-19 mortality and incidence in the United States.
DESIGN: Observational study.
SETTING: US county level case surveillance and vaccine administration data reported from 14 December 2020 to 18 December 2021. PARTICIPANTS: Residents of 2558 counties from 48 US states. MAIN OUTCOME MEASURES: The primary outcome was county covid-19 mortality rates (deaths/100 000 population/county week). The secondary outcome was incidence of covid-19 (cases/100 000 population/county week). Incidence rate ratios were used to compare rates across vaccination coverage levels. The impact of a 10% improvement in county vaccination coverage (defined as at least one dose of a covid-19 vaccine among adults ≥18 years of age) was estimated During the eras of alpha and delta variant predominance, the impact of very low (0-9%), low (10-39%), medium (40-69%), and high (≥70%) vaccination coverage levels was compared.
RESULTS: In total, 30 643 878 cases of covid-19 and 439 682 deaths associated with covid-19 occurred over 132 791 county weeks. A 10% improvement in vaccination coverage was associated with an 8% (95% confidence interval 8% to 9%) reduction in mortality rates and a 7% (6% to 8%) reduction in incidence. Higher vaccination coverage levels were associated with reduced mortality and incidence rates during the eras of alpha and delta variant predominance.
CONCLUSIONS: Higher vaccination coverage was associated with lower rates of population level covid-19 mortality and incidence in the US. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

As of 11 April 2022, 497 960 492 cases covid-19 and 6 181 850 covid-19 related deaths had been reported globally, and 80 260 092 covid-19 cases and 983 237 covid-19 related deaths had been reported in the United States.1 2 The US death toll recently surpassed the 1918 Spanish flu as the deadliest pandemic in recent history.3 In addition to covid-19 related deaths, the pandemic has also had indirect effects on other health conditions. These effects are captured in excess mortality and reduced life expectancy estimates. Domestically, life expectancy decreased by 1.5 years from 2019 to 2020, representing the largest reduction since the second world war.4

Messenger RNA (mRNA) covid-19 vaccines developed by Pfizer-BioNTech and Moderna and an adenovirus covid-19 vaccine developed by Johnson & Johnson have become valuable tools to combat this pandemic. Clinical trials evaluating efficacy against symptomatic infection found that the Pfizer-BioNTech vaccine was 95.0% effective, the Moderna vaccine was 94.1% effective, and the Janssen vaccine (Johnson & Johnson) was 66.3% effective.5 6 7 The US Food and Drug Administration (FDA) granted emergency use authorization for mRNA vaccines in December 2020 and the Janssen vaccine in February 2021. FDA approval for the Pfizer and Moderna vaccines was granted in August 2021 and January 2022, respectively.8 Emergency use authorization was further granted to additional doses of the mRNA vaccines for certain populations.8 As of 11 April 2022, nearly 570 million vaccine doses have been administered in the US and more than 11 billion vaccine doses have been administered globally.1 2 The World Health Organization’s target is to vaccinate 70% of the world’s population by mid-2022.9

Across countries, the real world effectiveness of the covid-19 vaccines has largely been consistent with estimates of efficacy observed in clinical trials.10 11 12 In addition to the individual level effect on disease risk and progression, vaccines may also have secondary benefits of slowing spread and reducing onward transmission and its associated morbidity and mortality.13 Population level data and analyses have been limited.14 15 We aimed to estimate how increasing county coverage of vaccines affected population level mortality and incidence of covid-19.

Methods

Study design

Our observational study of the US population used national, county level surveillance data. In the US, counties are a geographic administrative unit below states and territories and include the nation’s capital, Washington DC. The US Centers for Disease Control and Prevention (CDC) receives surveillance data from 3224 US counties (or county equivalents). We included and analyzed county covid-19 cases, deaths, and vaccinations reported to the CDC. We tracked mortality as our primary outcome and incidence (using reported probable and confirmed covid-19 cases) as our secondary outcome. We calculated county level incidence by standardizing reported county cases and deaths per 100 000 population over a week.2

Study definitions

We defined a case as one that met the Council of State and Territorial Epidemiologists’ surveillance case definitions as confirmed or probable covid-19 and a death as those that were related to covid-19, as determined or reported by jurisdictions.16 17 Each vaccine dose administered was attributed to the county in which the person resided.18 We defined the county vaccination coverage as the number of people aged ≥18 years who received at least one dose of covid-19 vaccine among the total number of people aged ≥18 years old residing in that county.2

Data sources

For case and death counts disaggregated by county and week, we used the CDC’s managed case surveillance dataset, which includes the most recent numbers reported by states, territories, and other jurisdictions. This dataset is populated by routine reporting from jurisdictions to the CDC.16 To document new cases, jurisdictions may use the date that a case was reported to the health department, a person took a covid-19 test, a laboratory confirmed a covid-19 test as positive, or a person was diagnosed as having covid-19 by a clinician. For death reporting, jurisdictions may use the date when the death was reported to the health department or the date of covid-19 associated death.2 We retrieved counts of covid-19 vaccine doses administered by week and county from the CDC’s managed vaccine dataset. This dataset includes covid-19 vaccination data (including the date of vaccine administration, the number of doses administered, and county of residence, among other variables) reported to the CDC by jurisdictions, pharmacies, and federal entities through Immunization Information Systems, the Vaccine Administration Management System, or direct submission of vaccination records.19 The population data, used for denominators to measure vaccination coverage, came from the vintage 2019 US population estimates.20

Inclusion criteria

We included case surveillance and vaccine administration data from 14 December 2020 to 18 December 2021. We included people at least 18 years of age with a valid county of residence in one of the states or territories who received at least one covid-19 vaccination. Given that population benefits may extend beyond the primary vaccine recipient, we included case and mortality data across all ages. Data completeness was an inclusion criterion for analysis. We used a 70% threshold for data completeness of reporting county of residence across all data sources. Specifically, we excluded a jurisdiction if more than 30% of the case, death, and/or vaccination data for the jurisdiction were contributed by unspecified or unknown counties of residence. We excluded Texas and Hawaii because vaccination data were unavailable at county level. We excluded county equivalents in territories except for Puerto Rico and Guam, either because the county level population data of adults ≥18 years old were unavailable (US Virgin Islands) or because the county equivalent vaccination data were unavailable (all other territories). In addition, we excluded eight counties in California with a population of fewer than 20 000 people, as California does not report the vaccination data of counties with under 20 000 people. We excluded the Kusilvak Census Area in Alaska owing to unavailable vaccination data and the Valdez-Cordova Census Area in Alaska because the case and mortality data were unavailable. We excluded the District of Columbia, villages in Guam, and municipalities in Puerto Rico because of a lack of mobility data. Finally, we excluded Rio Arriba County in New Mexico because the social vulnerability index was missing. In addition, we excluded any county week missing covariate information used in regression models.

Data analysis

County of residence case and first dose covid-19 vaccination data were aggregated by Morbidity and Mortality Weekly Report (MMWR) week beginning with MMWR week 2020-51 (13-19 December 2020) and ending with MMWR week 2021-50 (12-18 December 2021).21 The CDC and Agency for Toxic Substances and Disease Registry’s social vulnerability index encompasses socioeconomic status (that is, poverty rates, unemployment rates, income levels, and education levels), household composition and disability (that is, ages, disability, and single parent households), minority status, language capability, and housing type and transportation (that is, multi-unit structures, mobile homes, crowding levels, vehicle ownership, and group housing) into a single measure.22 Google’s community mobility reports help to measure changes in community mobility related to covid-19.23 To prevent confounding related to the social vulnerability and mobility of communities, we included these variables in the model.24 25

We used generalized linear mixed models assuming a negative binomial outcome distribution to assess associations between vaccination coverage and rates of deaths and cases by using continuous estimates.26 We used a first order autoregressive correlation structure to account for multiple observations per county and for potential autocorrelation. We included county level population as an offset and included social vulnerability index categorized into quarters and retail and work mobility data as covariates. To account for cases occurring during the period of developing immunity, a county remained in the lower vaccination category for two weeks before moving to the next vaccination category.

We calculated estimates during the period of alpha variant predominance and the period of delta variant predominance (starting when the national prevalence of delta was estimated to be at least 50%—that is, the week of 20 June 2021 onward) categorically.27 28 We compared four different categories for county vaccination coverage: very low (0-9% of the county had been vaccinated), low (10-39% of the county had been vaccinated), medium (40-69% of the county had been vaccinated), and high (≥70% of the county had been vaccinated) during the era of alpha variant predominance. As with the continuous analyses, to account for cases occurring during the period of developing immunity, a county remained in the lower vaccination category for two weeks before moving to the next vaccination category. Moreover, we included county level population as an offset and included social vulnerability index categorized by quarter and retail and work mobility data as covariates. Given the inadequate number of county weeks accrued with very low and low vaccination coverage, we compared the mortality and incidence rates for medium and high coverage during the era of delta variant predominance.

Sensitivity analyses

We did three sensitivity analyses with the continuous analyses. The first sensitivity analysis was to compare definitions of vaccination being at least one dose with including only fully vaccinated people (that is, at least two mRNA doses or a single adenovirus dose). The second was to compare use of a stringency level for data completeness of 70% and 90%. The third was to compare estimates with and without the two week lag period.

Patient and public involvement

We used routinely generated covid-19 vaccine and case surveillance data. No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results.

Results

First year of vaccine roll-out

We included data from 2558 counties in 48 US states (fig 1). In total, we observed 30 643 878 cases of covid-19 and 439 682 covid-19 related deaths over 132 791 county weeks (table 1). Every 10% improvement in vaccination coverage was associated with an 8% (95% confidence interval 8% to 9%) reduction in mortality rates (fig 2) and with a 7% (6% to 8% reduction in case incidence (fig 2).

Fig 1

County scale-up of vaccinations from 13 December 2020 to 18 December 2021. Light purple lines represent individual counties; dark purple line is aggregated estimate

Table 1

Characteristics of included counties. Values are median (range) unless stated otherwise

Characteristic	13 Dec 2020 to 18 Dec 2021	Alpha variant (13 Dec 2020 to 27 Jun 2021)	Delta variant (28 June 2021 to 18 Dec 2021)
Sample size (counties; weeks)	2558; 132 791	2557; 70 189	2543; 62 602
Vaccination coverage	46.4 (0.0-100.0)	24.9 (0.0-100.0)	58.8 (2.1-100.0)
Vaccination coverage, No (%):
0-9.9%	21 312 (16.0)	21 238 (30.3)	74 (0.1)
10-39.9%	31 838 (24.0)	28 138 (40.1)	3700 (5.9)
40-69.9%	65 473 (49.3)	19 513 (27.8)	45 960 (73.4)
≥70%	14 168 (10.7)	1300 (1.9)	12 868 (20.6)
Population size	24 538 (1074-7 894 557)	24 541 (1074-7 894 557)	24 696 (1404-7 894 557)
Social vulnerability index, No (%):
Quarter 1	620 (24.2)	620 (24.2)	614 (24.1)
Quarter 2	666 (26.0)	666 (26.0)	662 (26.0)
Quarter 3	655 (25.6)	654 (25.6)	652 (25.6)
Quarter 4	617 (24.1)	617 (24.1)	615 (24.2)
Adults aged ≥25 without high school diploma, %	11.8 (1.6-42.4)	11.8 (1.6-42.4)	11.8 (1.7-42.4)
Below federal poverty level, %	14.8 (2.3-55.1)	14.8 (2.3-55.1)	14.8 (2.3-55.1)
Per capita income, $	26 256 (10 148-72 832)	26 256 (10 148-72 832)	26 262 (10 148-72 832)
Unemployment rate	5.6 (0.7-25.8)	5.6 (0.7-25.8)	5.6 (0.7-25.8)
Age ≤17, %	22.3 (7.3-40.3)	22.3 (7.3-40.3)	22.3 (7.3-40.3)
Age ≥65, %	17.9 (3.8-55.6)	17.9 (3.8-55.6)	17.8 (3.8-55.6)
Age >5 with disability, %	15.5 (3.8-33.7)	15.5 (3.8-33.7)	15.5 (3.8-33.7)
Racial or ethnic minority, %	15.1 (0.3-95.7)	15.1 (0.3-95.7)	15.1 (0.3-95.7)
Single parent households, %	8.2 (1.9-25.6)	8.2 (1.9-25.6)	8.2 (1.9-25.6)
Limited English proficiency, %	0.7 (0.0-21.7)	0.7 (0.0-21.7)	0.7 (0.0-21.7)
Households without vehicle, %	5.8 (0.5-77.0)	5.8 (0.5-77.0)	5.8 (0.5-77.0)
Housing in structures with ≥10 units, %	3.2 (0.0-89.4)	3.2 (0.0-89.4)	3.2 (0.0-89.4)
In mobile homes, %	10.7 (0.0-54.8)	10.7 (0.0-54.8)	10.7 (0.0-54.8)
Occupied housing units where people exceed rooms, %	1.8 (0.0-35.4)	1.8 (0.0-35.4)	1.8 (0.0-35.4)
People in institutionalized group residencies, %	2.0 (0.0-36.2)	2.0 (0.0-36.2)	2.0 (0.0-36.2)
Change in mobility, %:
Groceries	4.6 (−91.0-206.7)	−0.7 (−91.0-140.0)	8.0 (−80.0-206.7)
Home	4.1 (−23.2-41.8)	6.0 (−4.0-41.8)	3.3 (−23.2-15.4)
Parks	33.0 (−84.6-490.0)	10.4 (−84.6-433.0)	58.7 (−81.3-490.0)
Retail	−1.4 (−88.0-304.9)	−7.1 (−88.0-163.4)	2.6 (−85.0-304.9)
Transit	−6.6 (−85.4-280.1)	−14.4 (−82.0-258.6)	3.1 (−85.4-280.1)
Offices	−18.4 (−85.4-65.7)	−18.4 (−79.8-32.4)	−18.4 (−85.4-65.7)

Fig 2

Effect of vaccination coverage on county covid-19 related mortality (top) and incidence (bottom) during first year of vaccine roll-out. Analyses are from 2558 counties in 48 US jurisdictions. Model controlled for county population size, social vulnerability index, and mobility changes

Era of alpha variant predominance

In total, we observed 15 493 299 cases of covid-19 and 263 873 covid-19 related deaths over 70 189 county weeks during the era of alpha variant predominance. Compared with very low coverage, low (incidence rate ratio 0.40, 95% confidence interval 0.39 to 0.42), medium (0.25, 0.23 to 0.26), and high (0.19, 0.16 to 0.22) vaccination coverage categories had lower rates of mortality (fig 3). Compared with very low coverage, low (incidence rate ratio 0.43, 0.41 to 0.44), medium (0.30, 0.29 to 0.32), and high (0.20, 0.18 to 0.22) vaccination coverage categories had lower incidence rates (fig 3).

Fig 3

Effect of vaccination coverage on county covid-19 mortality and incidence during era of alpha variant predominance. Analyses are from 2557 counties in 48 US jurisdictions. Model controlled for county population size, social vulnerability index, and mobility changes

Era of delta variant predominance

In total, we observed 15 150 579 cases of covid-19 and 175 809 covid-19 related deaths over 62 602 county weeks during the era of delta variant predominance. When comparing high and medium coverage, we observed similar mortality effects during the era of alpha variant predominance (incidence rate ratio 0.77, 0.66 to 0.90) and the era of delta variant predominance (0.75, 0.71 to 0.79). When comparing high and medium coverage, we observed smaller incidence effect sizes during the era of delta variant predominance (incidence rate ratio 0.90, 0.87 to 0.94) compared with the era of alpha variant predominance (0.66, 0.60 to 0.73).

We observed sustained reductions in county mortality and incidence rates when we included only fully vaccinated people in the vaccination coverage categories, when we increased our data stringency level, and when we removed the two week immunity lag period (fig 4).

Fig 4

Sensitivity analyses of including only fully vaccinated people, increasing data stringency requirements, and removing two week immunity lag period. *In baseline group, vaccination coverage refers to coverage of at least one dose of vaccine, 2558 counties and 48 US jurisdictions included had ≥70% completeness rates of reporting county of residence, and study period was 14 December 2020 to 18 December 2021. †Vaccination coverage refers to coverage of fully vaccinated people. ‡2164 counties and 42 US jurisdictions included had ≥90% completeness rates of reporting county of residence. §Two week immunity period was removed

Discussion

Using data from 2558 counties—representing nearly 300 million people and 80% of the US population—we found that increasing the vaccination coverage in counties was associated with a reduced incidence of covid-19 related mortality and cases. We observed decreasing trends in mortality and case incidence associated with higher levels of vaccination coverage across the eras of both alpha and delta variant predominance. This effect was robust to various sensitivity analyses, which improves prediction and confidence in these findings.

Covid-19 associated mortality remains one of the most important clinical outcomes to guide public health decision making, measure pandemic severity, and evaluate mitigation efforts. It was our primary outcome. In the US, death registration rates, and cause of death ascertainment, remain high. This suggests that US mortality surveillance systems have been, and will continue to be, useful for covid-19 mortality surveillance. Previous vaccine studies have shown survival benefits at the individual level.29 We observed that these benefits may extend to the population level; counties with high coverage had a greater than 80% reduction in mortality rates compared with largely unvaccinated counties. Given that infection fatality rates for covid-19 increase with age, counties with a higher proportion of older people may have more covid-19 related mortality and stand to benefit from high coverage of covid-19 vaccines.30

We used reported cases as a proxy for incidence for our secondary outcome. Although reliable, available across jurisdictions, and reported continuously, reported cases may not reflect true transmission rates because of variation in when people seek out testing.31 For example, people without symptoms may not actively seek out testing of their own accord but may be important to test for gauging disease transmission. Owing to more recent reopening requirements for workplaces, restaurants, entertainment venues, schools, and outgoing international air travel, more people without symptoms may be seeking out testing.32 These requirements, and their uptake, may vary across states and counties. Nevertheless, the reduction in incidence observed with increasing vaccination coverage is consistent with surveillance data from other countries that have achieved high vaccination coverage and emerging evidence on transmission from contact tracing programs.1 33

Comparison with other studies

Increasing vaccination coverage may play a role in mitigating the effects of the delta and omicron variants and reduce the emergence of future variants.34 35 By 27 June 2021 the delta variant made up more than 50% of circulating variants in the US, increasing to almost 100% by 21 September 2021.2 More recently, the omicron variant was first reported on 1 December 2021 and comprised 95% of circulating variants by 2 January 2022.2 The delta variant had increased transmissibility and possible increased virulence compared with earlier SARS-CoV-2 strains.36 37 In our study, by the time the delta variant predominated, counties with lower levels of vaccination coverage (that is, 0-39%) were rare. Nevertheless, our findings of continued population level protection against death and reductions in population level protection against infection during the period of delta variant predominance seem to be consistent with clinical literature on vaccine effectiveness.29 38 39 40 Additional studies aimed at assessing the population impact of vaccines during the period of delta variant predominance merit consideration for validating our observations. Although our study period did not include the period of omicron variant predominance, data suggesting reduced vaccine effectiveness and the importance of staying up to date on covid-19 vaccinations are emerging and may lead to changes in population level vaccine impact that merit exploration.41 42 43 Continuing to monitor the delta and omicron variants, and the emergence of other variants of interest, is critical and will require ongoing genomic surveillance.

Clinical studies indicate that a single dose of an mRNA vaccine provides a lower level of protection than two doses.44 Furthermore, two mRNA doses seem to be more effective than a single adenovirus dose against symptomatic infection.5 6 7 We defined people with at least one dose of vaccine as being vaccinated for the purposes of vaccination coverage. Given that our study design used population surveillance data, changing our coverage definition to include only fully vaccinated people would place people with a single dose of mRNA vaccine in our referent, the very low coverage category. This may introduce bias in the incidence and mortality estimates. When we changed our definition of vaccination coverage to being fully dosed during sensitivity analysis, we did not find increased effect sizes, as would be expected from clinical studies.44 Ongoing vaccine studies continue to evaluate the comparative effectiveness of vaccines by manufacturer.10

Given that only people aged 18 and older were eligible for vaccination across vaccines during most of our study period, we used this age threshold to define vaccination coverage. Pediatric studies will be a welcome contribution to understanding the effects of vaccines on younger age groups, when feasible. As of September 2021 the FDA began recommending a third dose for specific populations.8 Owing to waning immunity and variant induced changes in vaccine effectiveness, additional doses may be needed in specific populations and scenarios. Further studies may benefit from evaluating the population impact of vaccination coverage by using different definitions and eligibility scenarios.

Limitations of study

Several limitations should be considered when interpreting these data. We chose vaccination coverage thresholds based on programmatic experience; exploring coverage thresholds above 70% may be worth examining in future research once more counties have achieved these levels for extended periods of time. We excluded some jurisdictions because they did not have county level information on immunizations, cases, and deaths for at least 70% of their counties. Additional markers of disease severity, such as hospital admissions, were not explored in this study owing to possible differences in ascertainment and reporting coverage across jurisdictions. Given the limited number of variables that were known to affect mortality and incidence, collected at the county level, and available on a weekly basis, we did not control for masking, physical distancing, or other similar potential confounding variables in this study. Furthermore, given the limited number of county weeks, we lacked power to stratify by time periods and cannot rule out the possibility of temporal confounding. Finally, given that we used aggregate case surveillance data to have the most complete case and death data available, other characteristics of cases, such as demographics and comorbidities, were not available. States, territories, and jurisdictions adapt national guidance on which date to use for case reporting.17 In this study we collated county data across these geographic areas. A time difference may be present depending on which date a health department uses; however, this is unlikely to be substantial enough to affect which week a case or death occurs. Naturally acquired immunity resulting from SARS-CoV-2 infection may have affected the reduced case incidence observed during the study period. These are limitations of our study. Nevertheless, reductions in incidence and death observed in emerging US data using alternative data sources and study designs give us confidence in the directionality and magnitude of our estimates.45 46

Conclusions and policy implications

In addition to individual level benefits, we observed that vaccines protect communities against severe disease and infection. Higher coverage of vaccines seemed to confer greater levels of community benefits. Given that community benefits are rooted in individual benefits, for which vaccine effectiveness has been established in countries around the world, these data may be generalizable to other countries. Future research may benefit from evaluating macroeconomic effects of improving population health, such as changes in employment rates and gross domestic product resulting from reopening society. Vaccines should be deployed strategically with public health and social measures based on ongoing levels of transmission.

The public health impact of scaling up covid-19 vaccination remains largely uncharacterized

Higher vaccination coverage was associated with lower rates of population level incidence of covid-19 and mortality related to covid-19

This community level benefit complements the large body of evidence indicating individual level benefits of covid-19 vaccination