Comparative vaccine effectiveness against severe COVID-19 over time in US hospital administrative data: a case-control study.

BACKGROUND: Research suggests the protection offered by COVID-19 vaccines might wane over time, prompting consideration of booster vaccinations. Data on which vaccines offer the most robust protection over time, and which patients are most vulnerable to attenuating protection, could help inform potential booster programmes. In this study, we used comprehensive hospitalisation data to estimate vaccine effectiveness over time.
METHODS: In this case-control study, we used data from a large US health-care system to estimate vaccine effectiveness against severe SARS-CoV-2 infection and examined variation based on time since vaccination, vaccine type, and patients' demographic and clinical characteristics. We compared trends in attenuation of protection across vaccines and used a multivariable model to identify key factors associated with risk for severe breakthrough infection. Patients were considered to have severe COVID-19 if they were admitted to the hospital, had a final coded diagnosis of COVID-19 (according to International Classification of Diseases Tenth Revision code U07.1) or a positive nucleic acid amplification test for symptomatic SARS-CoV-2 during their hospitalisation, and were treated with remdesivir or dexamethasone during hospitalisation.
FINDINGS: Between April 1, 2021, and Oct 26, 2021, we observed 9667 admissions for severe COVID-19 (ie, cases). Overall, 1293 (13·4%) of 9667 cases were fully vaccinated at the time of admission, compared with 22 308 (57·7%) of 38 668 controls, who were admitted to hospital for other reasons. The median time between vaccination and hospital admission among cases was 162 days (IQR 118-198). Overall vaccine effectiveness declined mostly over the course of the summer, from 94·5% (95% CI 91·4-96·5) in April, 2021 (pre-delta), to 84·0% (81·6-86·1) by October, 2021. Notably, vaccine effectiveness declined over time, from 94·0% (95% CI 92·8-95·0) at days 50-100 after vaccination to 80·4% (77·8-82·7) by days 200-250 after vaccination. After 250 days, vaccine effectiveness declines were even more notable. Among those who received the BNT162b2 (Pfizer-BioNTech) vaccine, vaccine effectiveness fell from an initial peak of 94·9% (93·2-96·2) to 74·1% (69·6-77·9) by days 200-250 after vaccination. Protection from the mRNA-1273 (Moderna) and Ad26.COV2 (Janssen) vaccines declined less over time, although the latter offered lower overall protection. Holding other factors constant, the risk of severe breakthrough infection was most strongly associated with age older than 80 years (adjusted odds ratio 1·76, 95% CI 1·43-2·15), vaccine type (Pfizer 1·39, 0·98-1·97; Janssen 14·53, 8·43-25·03; both relative to Moderna), time since vaccination (1·05, 1·03-1·07; per week after week 8 when protection peaks, technically), and comorbidities including organ transplantation (3·44, 95% CI 2·12-5·57), cancer (1·93, 1·60-2·33), and immunodeficiency (1·49, 1·13-1·96).
INTERPRETATION: Vaccination remains highly effective against hospitalisation, but vaccine effectiveness declined after 200 days, particularly for older patients or those with specific comorbidities. Additional protection (eg, a booster vaccination) might be warranted for everyone, but especially for these populations. In addition to promoting general vaccine uptake, clinicians and policy makers should consider prioritising booster vaccinations in those most at risk of severe COVID-19. FUNDING: None.

Introduction

Concern about waning COVID-19 vaccine effectiveness has prompted intense public discussion about the need for booster vaccinations, particularly given the rise of the SARS-CoV-2 delta (B.1.617.2) variant. Pre-delta variant studies showed that the vaccines had high effectiveness,1, 2, 3 which remained largely intact for up to 6 months. Early research on the delta variant suggested only a modest difference in vaccine effectiveness, but subsequent reports suggested protection against delta infection might be somewhat lower than previous variants.6, 7, 8, 9, 10, 11 A US study found vaccine effectiveness fell from 92% to 80% as delta cases rose, although vaccine effectiveness against hospitalisation remained at 90%. Another study that focused on patients in nursing homes saw vaccine effectiveness decline from 75% to 53% as delta became the dominant US SARS-CoV-2 variant, whereas a third study found vaccine effectiveness against hospitalisation fell from 90% to 63% during the rise of the delta variant.

Evidence before this study

Previous studies have used retrospective case-control designs to assess vaccine effectiveness against severe SARS-CoV-2 infection requiring hospitalisation. Results have shown strong overall protection, but some indications that protection might begin to wane as patients approach 6 months post-vaccination. These results have given rise to substantial discussion about the merits of potential booster shots. We searched MEDLINE between March 1, 2020, to Nov 15, 2021, using the terms “covid-19 vaccine studies”, “covid-19 vaccine effectiveness”, and “covid-19 vaccine efficacy”. We considered only publication in English or with English translations.

Added value of this study

We used a case-control design similar to other published studies and found similar overall estimates of vaccine effectiveness. We added three distinct points of additional value. First, we extended existing studies to examine potential attenuation of protection beyond 6 months, finding evidence of significant declines in protection after 200 days post-vaccination for some patients. Second, we examined whether different vaccines differed in terms of waning protection over time, finding that the BNT162b2 (Pfizer-BioNTech) vaccine is particularly susceptible to declines after 200 days, whereas the mRNA-1273 (Moderna) and Ad26.COV2 (Janssen) vaccines are more robust over time, although the Janssen vaccine offers lower overall vaccine effectiveness. Finally, we found that, keeping other factors constant in a multivariable model, the risk of having a severe breakthrough infection was most strongly associated with age older then 80 years, vaccine type, time since vaccination, and specific comorbidities, including cancer, organ transplantation, chronic kidney disease, hypertension, or heart failure.

Implications of all the available evidence

Our results could help with purchasing and prioritising decisions for any potential booster vaccine programme. There are important differences in performance over time by vaccine type, and important differences in the risks of severe breakthrough infection by patient type, which should inform policy, programmatic, and clinical decision making around booster vaccinations.

More recent data suggest that declining vaccine effectiveness might be driven more by waning protection than the delta variant of SARS-CoV-2. A large retrospective 6-month study of the mRNA BNT162b2 (Pfizer-BioNTech) vaccine found that effectiveness declined after 4–5 months against both delta and non-delta SARS-CoV-2 infections, although protection against hospitalisations remained high for 6 months. Other studies have shown similar evidence of declining effectiveness against infection, but robust protection against severe illness, over the first 6 months after vaccination. Such reports of declining effectiveness have ramifications for international vaccine policy; Israel has announced its intention to administer booster vaccinations to citizens older than 12 years, the USA has updated its initial authorisation, which targeted adults at enhanced risk, to encompass boosters for all adults, and several European countries are also initiating booster programmes.18, 19, 20 These moves potentially open the door to widespread global use of booster vaccinations, despite concerns from WHO about equity and potentially exacerbating global vaccine supply constraints.

Nations grappling with the question of booster vaccines need answers to some crucial questions to help inform their decisions. Evidence of robust protection against hospitalisations is encouraging, but data are mostly limited to studies of the first 6 months or less, with very little data on what happens after 6 months. Although some studies show variation in vaccine effectiveness by vaccine type, little is understood about how changes in protection over time differ between vaccine types—which could help inform purchase decisions—or for distinct types of patients—which could help identify priority populations for potential booster programmes.

In this study, we used comprehensive hospitalisation data to estimate vaccine effectiveness over time using a case-control design similar to other published research.10, 13, 14, 22 We add several new insights. First, we extended our analysis of waning protection to 250 days (over 8 months) after vaccination, allowing us to examine attenuation of protection against severe COVID-19 over a longer time period. Second, we examined patterns of declining effectiveness by vaccine type to help inform investment decisions for nations considering booster programmes. Finally, we used multivariable modelling to identify the key factors associated with risk of severe SARS-CoV-2 infection in fully vaccinated individuals and applied our findings to inform optimal prioritisation and deployment of booster vaccines.

Methods

Overview and objectives

In this study, we used comprehensive hospital admission data from a large US health-care system (Providence) to analyse vaccine effectiveness against severe COVID-19 over time using a case-control design. To help understand how protection changed over time and inform decisions about potential booster programmes, we examined variation in vaccine effectiveness over time by vaccine type and specified a multivariable model to identify key individual factors that shape the risk of a vaccinated person having a severe breakthrough COVID-19 infection.

This study adhered to the Strengthening the Reporting of Observational Studies in Epidemiology guidelines, and was approved by the Providence Institutional Review Board (study #2021000504), which authorised a waiver of consent for this retrospective records review.

Data sources and variables

Hospital inpatient encounter records were obtained from electronic medical records covering 42 hospitals in six western US states over a 7-month period (April 1 to Oct 26, 2021). These hospitals are in Alaska, Washington, Oregon, California, Montana, and Texas and serve a catchment of about 6 million individuals. April 1, 2021, was selected as a start date because there were very few earlier COVID-19 hospitalisations of vaccinated patients before this date. Patients younger than 18 years or admitted for elective inpatient care were excluded from the study. Data collected on study patients were admission date, diagnoses, demographics, comorbidities, COVID-19 status (collected at admission for all patients), and vaccination status.

Patients were considered to have severe COVID-19 if they were admitted to the hospital, had a final coded diagnosis of COVID-19 (according to International Classification of Diseases Tenth Revision [ICD-10] code U07.1) or a positive nucleic acid amplification test for symptomatic SARS-CoV-2 during their hospitalisation, and were treated with remdesivir or dexamethasone (used within Providence exclusively for patients who require supplemental oxygen) during hospitalisation.

Vaccination data were obtained from electronic medical records and included vaccines administered by the health-care system, patient-reported vaccinations, and integrated data from comprehensive state vaccination registries. Full vaccination date was defined as 14 days after the final vaccination in a recognised series (BNT162b2 [Pfizer-BioNTech], mRNA-1273 [Moderna], or Ad26.COV2 [Janssen]), and time since vaccination was defined as the number of days between the most recent vaccine dose and the index admission date. We also captured vaccination type. We excluded patients who had received more than one type of vaccine, or who had been partly vaccinated. We also excluded patients who had received booster vaccinations before their index encounter. Patients were flagged if they had a previous COVID-19 diagnosis (ICD-10 codes U07.1 or Z86.16) or positive nucleic acid amplification test 50 days or more before the index admission. Severe breakthrough infections were defined as a severe SARS-CoV-2 infection for any patient with full vaccination status.

Comorbidities for each patient were identified via ICD-10 clinical modification billing diagnosis codes from the index encounter and the 365 days preceding the encounter. Codes were grouped into Agency for Healthcare Research and Quality Clinical Classifications Software-Refined categories for analysis. Other patient-level variables were age at admission, race, sex, date of admission, body-mass index, and smoking status.

We did not have individual sequencing data for each COVID-19 case in our study, so we used the US Centers for Disease Control and Prevention (CDC) weekly estimates of delta prevalence across United States Department of Health and Human Services regions as a proxy for the effect of delta on outcomes. For each geographical region in our analysis, the daily prevalence of delta at the time of each admission was linearly interpolated from the weekly CDC summary as a weighted average of the midpoint of the adjacent weeks. We then flagged all admissions that occurred during periods of high delta prevalence within their relevant geography (defined as 75% or more of all reported cases in the region).

Case-control design

We applied a case-control design, with patients admitted to the hospital for severe COVID-19 constituting cases, and a 4:1 matched group of patients admitted within 5 days and at the same hospital for any other non-elective reason constituting controls. Similar designs have been used to estimate vaccine effectiveness in previous studies.

Similar case-control studies have used different approaches to identify severe COVID-19 cases or matched controls. We preferred our approach because it allowed for a better match on date and site of care, which is important for separating out the effect of time since vaccination and delta variant prevalence. However, as a sensitivity analysis, we replicated our analyses using several alternative specifications with different methods of defining severe COVID-19 admissions, vaccination status, and matching to control cases, with results available in the appendix (pp 8–19).

Approach for calculating vaccine effectiveness

Vaccine effectiveness was calculated for any given subgroup as follows:

where a g is the count of COVID-19 admissions with full vaccination, b g is the count of non-COVID-19 admissions with full vaccination, c g is the count of COVID-19 admissions with no vaccination, and d g is the count of non-COVID-19 admissions with no vaccination. Vaccine effectiveness is equal to 1 minus the odds ratio (OR), which we used instead of rate ratios because they are scale independent, which helps reduce bias during virus surges.

Case-control logistic regression analysis

We used multivariable conditional logistic regression analysis to identify factors associated with severe breakthrough infection risk. Our equation took the stylised form:

where i indexes a set of patients and j indexes an individual within the set, Yij indexes the outcome of severe COVID-19 admission for an individual, and Xij indexes the values of the covariates for that individual. βvax is the coefficient for the OR of vaccination, βintx is the coefficient matrix for the variables interacting with vaccination status and thus represents the effects of those variables on vaccine effectiveness, and βadj is the coefficient matrix for the variables not interacting with vaccination and is interpreted as adjusting the odds of admission to hospital with severe COVID-19. Under this approach, each covariate (eg, asthma) is assessed both in terms of its direct effect on the outcome (probability of a severe COVID-19 admission) and its interaction with vaccine status (modifying the effect of vaccine status on probability of severe COVID-19 admission). The latter terms are our primary variables of interest in this paper. Once the model was fit, we isolated the interaction coefficients to calculate estimates of vaccine effectiveness conditional on any given set of covariate values.

Generating CIs for vaccine effectiveness estimates

We used bootstrapping to produce 95% CIs around our vaccine effectiveness estimates. Under this approach, we created multiple (n=1000) distinct samples from our single set of observations and used the resampling distribution to construct a 95% CI around our mean effect estimates.

Role of the funding source

There was no funding source for this study.

Results

Between April 1, 2021, Oct 26, 2021, we observed 9667 admissions for severe COVID-19 (ie, cases). Overall, 1293 (13·4%) of 9667 cases were fully vaccinated at the time of admission, compared with 22 308 (57·7%) of 38 668 controls. The median time between vaccination and hospital admission among cases was 162 days (IQR 118–198). Table 1 shows the composition of our sample, including cases (hospital admissions for severe COVID-19) and controls (hospital admissions for any other reason, matched by date and hospital in a 4:1 ratio to cases).

Table 1

Characteristics of cases and controls

		Cases (n=9667)	Matched controls (n=38 668)
Age, years
	Mean (SD)	57·5 (16·6)	61·6 (19·6)
	70–79	1490 (15·4 %)	7775 (20·1 %)
	≥80	964 (10·0%)	7629 (19·7 %)
Sex
	Female	4121 (42·6%)	20 127 (52·1%)
	Male	5546 (57·4%)	18 541 (47·9%)
Race or ethnicity
	White	6280 (65·0%)	27 730 (71·7%)
	Latino or Latina	1631 (16·9%)	4579 (11·8%)
	Black	372 (3·8%)	1652 (4·3%)
	Asian	334 (3·5%)	1643 (4·2%)
	Other	1050 (10·9%)	3064 (7·9%)
Vaccination and COVID-19 status
	Vaccinated	1293 (13·4%)	22 308 (57·7%)
	mRNA-1273 (Moderna)	370 (3·8%)	10 119 (26·2%)
	Ad26.COV2 (Janssen)	279 (2·9%)	2018 (5·2%)
	BNT162b2 (Pfizer-BioNTech)	634 (6·6%)	9882 (25·6%)
	Previously had COVID-19	53 (0·5%)	1491 (3·9%)
	Days since last vaccination	162 (118–198)	141 (92–178)
Comorbidities
	Alcohol or drug dependency	1525 (15·8%)	12 533 (32·4%)
	Asthma	1143 (11·8 %)	4084 (10·6%)
	Cancer	744 (7·7%)	6668 (17·2%)
	Heart failure	1304 (13·5%)	10 281 (26·6%)
	Chronic kidney disease	1395 (14·4%)	9739 (25·2%)
	Cognitive disease	510 (5·3%)	4352 (11·3%)
	Chronic obstructive pulmonary disease	1039 (10·7%)	6653 (17·2%)
	Coronary artery disease	1539 (15·9%)	11 189 (28·9%)
	Diabetes	3043 (31·5%)	12 493 (32·3%)
	Hypertension	4997 (51·7%)	25 403 (65·7%)
	Immunodeficiency	479 (5·0%)	1248 (3·2%)
	Obesity (body-mass index ≥35 kg/m2)	2773 (28·7%)	6854 (17·7%)
	Rheumatological disease	362 (3·7%)	1849 (4·8%)
	Smoker (current)	713 (7·4%)	6300 (16·3%)
	Organ transplantation	117 (1·2%)	476 (1·2%)
High delta prevalence (regional prevalence of ≥75%)		7686 (79·5%)	30 732 (79·5%)

Data are n (%) or median (IQR), unless otherwise indicated. Data are from Providence Electronic Medical Records on inpatient hospital encounters for people aged ≥18 years on April 1 to Oct 26, 2021. Cases include all severe COVID-19 admissions; controls are a 4:1 comparison group of non-elective admissions matched on date of admission and hospital of admission.

Overall vaccine effectiveness declined mostly over the course of the summer, from 94·5% (95% CI 91·4–96·5) in April, 2021 (pre-delta), to 84·0% (81·6–86·1) by October, 2021 (figure 1 ). Vaccine effectiveness was above 90% for all ages younger than 70 years, but as low as 72% (95% CI 62·3–79·2) for those aged 90 years and older.

Figure 1

Vaccine effectiveness against severe COVID-19 by encounter date (A) and patient age (B)

Graphs show blended vaccine effectiveness estimates of all vaccine types. All data are from Providence Electronic Medical Records on inpatient hospital encounters for individuals aged ≥18 years on April 1–Oct 26, 2021. Data comprise 9667 cases and 38 668 controls. Red lines represent mean vaccine effectiveness averages based on a rolling 30-day window for encounter data and a rolling 10-year window for patient age. The shaded areas represent 95% CIs.

Similar to previous studies, we found that protection peaks about 50 days after completing vaccination and begins to wane slightly after about 100 days (figure 2A ). Notably, we observed evidence of an accelerated decline in protection after 200 days, with overall vaccine effectiveness falling from 94·0% (95% CI 92·8–95·0) in days 50–100 to 80·4% (77·8–82·7) in days 200–250. We observed evidence of continued decline after 250 days, albeit with wider CIs because of fewer observations in this range. We found that across age groups and delta prevalence levels, protection against delta was uniformly lower across all time periods, but that attenuation of protection over time remained evident, especially in older patients (figure 2B). We observed especially stark reductions for those aged 80 years and older after 200 days since vaccination, with vaccine effectiveness falling to 78·4% (95% CI 73·6–82·4) 200–250 days after vaccination for this age group.

Figure 2

Vaccine effectiveness against severe COVID-19 by time since vaccination (A) and by time since vaccination and age (B)

Graphs show adjusted vaccine effectiveness estimates for all vaccine types. All are data from Providence Electronic Medical Records on inpatient hospital encounters for individuals aged ≥18 years on April 1–Oct 26, 2021. Data comprise 9667 cases and 38 668 controls. In part (A), the red line represents adjusted mean vaccine effectiveness based on a rolling 30-day window. Time since vaccination was computed as the number of days between a patient's most recent dose and the index admission. The shaded areas represent 95% CIs.

Overall, the mRNA-1273 (Moderna) vaccine showed the highest overall vaccine effectiveness across our study period, starting at 97·3% (95% CI 96·0–98·2) in days 50–100 after vaccination, then falling to 87·6% (84·5–90·1) by days 200–250, an overall decline of 9·7 percentage points (figure 3 ). The vaccine effectiveness of BNT162b2 (Pfizer-BioNTech) began similarly to mRNA-1273 at 94·9% (93·2–96·2), but showed substantially greater attenuation, falling to 74·1% (69·6–77·9) by 200–250 days, a decline of 20·8 percentage points. Finally, the Ad26.COV2 (Janssen) vaccine offered lower initial protection (78·5% at days 50–100, 95% CI 71·3–83·9) but showed less substantial attenuation over time, remaining at 72·6% (53·7–83·8) by days 200–250, a decline of 5·9 percentage points.

Figure 3

Vaccine effectiveness against severe COVID-19 by time since vaccination and vaccine type

All data are from Providence Electronic Medical Records on inpatient hospital encounters for individuals aged ≥18 years on April 1–Oct 26, 2021. Data comprise 9667 cases and 38 668 controls. Time since vaccination is computed as the number of days between a patient's most recent dose and the index admission. Shaded areas represent 95% CIs.

When considering adjusted ORs from our case-control regression analysis, differences by vaccine type remain apparent (table 2 ). Relative to those who received the mRNA-1273 (Moderna) vaccine and holding other factors constant, patients who received the Ad26.COV2 (Janssen) vaccine had substantially higher relative odds of hospitalisation for severe COVID-19 (adjusted OR 14·53, 95% CI 8·43–25·03). Patients who received the BNT162b2 (Pfizer-BioNTech) vaccine also had somewhat higher relative odds or hospitalisation, although the difference was not significant (1·39, 0·98–1·97). Time since vaccination also mattered. After peak protection at day 56 (8 weeks) after vaccine course completion, each subsequent week saw the relative odds of a severe breakthrough infection increase by around 5% (adjusted OR 1·05, 95% CI 1·03–1·07). Results also suggest that the delta variant was an important driver of severe breakthrough infections. Holding weeks since vaccination and other factors constant, 100% delta prevalence doubled the relative odds of a severe breakthrough infection compared with zero delta conditions (adjusted OR 2·00, 95% CI 1·35–2·94).

Table 2

Logistic regression analysis to identify factors associated with severe breakthrough COVID-19 infections in fully vaccinated individuals

		Adjusted OR (95% CI)	p value
Age, years
	70–79 (vs <70 reference)	1·11 (0·93–1·33)	0·26
	≥80 (vs <70 reference)	1·76 (1·43–2·15)	<0·0001
Male (vs female referent)		1·03 (0·90–1·18)	0·69
Race or ethnicity
	Black (vs White reference)	1·39 (0·97–1·99)	0·075
	Latino or Latina (vs White reference)	1·10 (0·89–1·34)	0·38
	Asian (vs White reference)	0·82 (0·59–1·13)	0·22
	Other (vs White reference)	1·09 (0·86–1·39)	0·46
Vaccination status
	BNT162b2 (Pfizer-BioNTech; vaccination with mRNA-1273 [Moderna] as reference)	1·39 (0·98–1·97)	0·067
	Ad26.COV2 (Janssen; vaccination with mRNA-1273 [Moderna] as reference)	14·53 (8·43–25·03)	<0·0001
Vaccine timing*
	Weeks before full protection at week 8	1·29 (1·16–1·44)	<0·0001
	Weeks after full protection at week 8	1·05 (1·03–1·07)	<0·0001
Comorbidities†
	Alcohol or drug dependency	1·81 (1·47–2·23)	<0·0001
	Asthma	0·95 (0·78–1·16)	0·62
	Cancer	1·93 (1·60–2·33)	<0·0001
	Chronic kidney disease	1·41 (1·19–1·67)	0·0010
	Chronic obstructive pulmonary disease	1·03 (0·86–1·23)	0·77
	Cognitive disease	1·25 (0·99–1·57)	0·059
	Coronary artery disease	1·05 (0·89–1·24)	0·53
	Diabetes	1·12 (0·97–1·30)	0·11
	Heart failure	1·47 (1·24–1·75)	<0·0001
	Hypertension	1·43 (1·20–1·71)	0·0018
	Immunodeficiency	1·49 (1·13–1·96)	0·0052
	Obesity (body-mass index ≥35 kg/m2)	0·82 (0·69–0·96)	0·016
	Rheumatological disease	1·52 (1·16–2·01)	0·0027
	Smoker (current)	1·30 (0·96–1·77)	0·091
	Organ transplantation	3·44 (2·12–5·57)	<0·0001
All delta (vs no delta reference)		2·00 (1·35–2·95)	0·0002

All data are from Providence Electronic Medical Records on inpatient hospital encounters for people aged ≥18 years on April 1 to Oct 26, 2021 (n=9667 cases, 38 668 controls). Output produced via multivariable logistic regression using the formula provided in the Methods. Displayed adjusted ORs represent the interaction terms of each relevant factor with vaccine status—that is, the adjusted odds of a severe COVID-19 infection contingent on being vaccinated—rather than the direct effect of each factor on risk of COVID-19 admission. Full model output is available in the appendix (pp 23–24). OR=odds ratio.

Reference is week 8 after vaccination completion, which is the point of peak protection.

For comorbidities, each condition is a binary so reference is absence of that condition.

The risk of severe breakthrough infections was not uniform by patient type. Holding other factors constant, patient comorbidities associated with the highest risk of a severe breakthrough infection included organ transplantation (adjusted OR 3·44, 95% CI 2·12–5·57), cancer (1·93, 1·60–2·33), and immunodeficiency (1·49, 1·13–1·96), whereas some conditions often associated with increased risk of severe breakthrough infections, such as coronary artery disease (1·05, 0·89–1·24) or chronic obstructive pulmonary disease (1·03, 0·86–1·23) were found not to be. Age was also a key predictor of risk, with patients aged 80 years and older having substantially higher relative odds of severe breakthrough infection than those younger than 70 years (1·76, 1·43–2·15). We found no evidence of differential risk by race or sex in our primary analysis.

Discussion

In this study, we used comprehensive hospital admission data to analyse the risk of severe COVID-19 in previously vaccinated individuals, with the goal of understanding differences in waning protection by vaccine type and identifying the key risk factors for severe breakthrough infections to inform the targeting of potential vaccine booster programmes. Consistent with other studies, we found that vaccine effectiveness against hospitalisation began to decline after 6 months, especially for older patients. Unlike most other studies, our data were able to stretch beyond 6 months, after which we found evidence of rapidly waning protection, especially for patients aged 80 years or older. We were able to identify important differences by vaccine type and patient characteristics that could help inform potential booster programmes.

Overall, we found that vaccines were less effective against severe COVID-19 by October, 2021, compared with April, 2021. Vaccine effectiveness was steady up to June, 2021, then began to decline in July, 2021, as delta prevalence increased and many patients became further removed from their vaccination dates.

The median time since full vaccination among cases in our study was about 5 months (162 days). Most patients had only modest declines in protection over the first 6 months since their final dose. However, we found evidence of accelerated decline after 200 days.

All vaccines showed evidence of strong continuing protection against hospitalisation, but there were differences in how their protection changed over time. Patients who received mRNA-1273 (Moderna) had the highest overall protection and saw relatively modest attenuation over time. By contrast, patients who received the BNT162b2 (Pfizer-BioNTech) vaccine began with similar levels of protection, but this protection diminished more substantially over time. The Ad26.COV2 (Janssen) vaccine offered less initial protection, but vaccine effectiveness remained mostly steady over time. Our Janssen estimates were based on fewer observations and had wider 95% CIs than did those for the Pfizer-BioNTech and Moderna vaccines, so should be interpreted with caution.

Attenuating vaccine protection over time is related to age. Overall, vaccine effectiveness remained at or above 90% until age 70, after which we observed increasing declines in effectiveness with age. Importantly, all age groups had only modest loss of protection against hospitalisation over the first 200 days after vaccination, but after 200 days we observed evidence of eroded protection, especially among those aged 70 years and older. We found a particularly sharp drop in vaccine effectiveness for patients aged 80 years and older after 200 days since vaccination. Even without this age difference, vaccine effectiveness declines in general are more dangerous for older patients; a decline in protection for patients aged 80 years and older will probably lead to many more hospitalisations and deaths compared with a similar decline in younger people, given that older age had been associated with more severe COVID-19 outcomes. Overall our findings suggest an urgent need for booster shots among older people who are 200 days or more from their initial vaccination date.

As expected,28, 29, 30 disorders associated with immunocompromise increased the risk of severe breakthrough infections, including cancer, chronic kidney disease, and organ transplantation. Less easily explained is the association of certain other chronic diseases with lower vaccine effectiveness, such as hypertension and heart failure, but not others, such as diabetes, coronary artery disease, or obesity. Further research might be needed to explore the mechanisms of interaction between chronic diseases and vaccine protection. Overall, our results suggest that patients with immune compromising disorders, hypertension, and heart failure could be the highest priority for targeted booster vaccination campaigns.

Overall, our data show strong protection against hospitalisation over the first 6 months after vaccination for most people, but evidence of waning protection after 200 days, particularly for patients aged 70 years and older. We found evidence that vaccine effectiveness is lower for patients with immune compromising conditions, hypertension, or heart failure. Finally, our data suggest that the mRNA-1273 (Moderna) vaccine offers the strongest and most robust protection over time. A series of sensitivity analyses (appendix pp 8–18) testing alternative specifications and matching strategies largely supported our overall findings.

Our findings can be contextualised against several other real-world studies of vaccine effectiveness. A large study of Swedish registries found an even stronger decline in vaccine effectiveness against hospitalisation after 180 days, albeit with a different mix of vaccine types, and a systematic review found that on average across the literature, vaccine effectiveness against severe disease decreased by 9·7 percentage points (95% CI 5·9–14·7) over the first 6 months after vaccination, roughly in line with our estimate of 7·5 percentage points by days 150–200. An Italian health ministry report found less evidence for declining vaccine effectiveness at 6 months, but noted the beginnings of potential decline in the face of increasing delta variant cases at the end of their follow-up period. Furthermore, several studies have found similar evidence for age and comorbidities as key drivers of risk for breakthrough infections or severe breakthrough infections.34, 35

Our study has several key limitations. First, our data are regionally concentrated and might not be nationally representative of all COVID-19 admissions and do not include results for children younger than 18 years. We also lacked individual sequencing data for infections, and thus relied on a proxy (regional prevalence) to estimate the influence of the delta variant on outcomes. We were only able to examine vaccine effectiveness against severe disease, but infections that do not result in a hospital admission might still have serious health consequences. Like other observational vaccine effectiveness studies, our analysis assumed a similar risk of exposure to COVID-19 for vaccinated and unvaccinated people, and we were unable to assess other factors, such as masking, distancing, or other social mitigation strategies that might themselves be associated with vaccine uptake.

In conclusion, this study of large-scale hospitalisation data in a major US health-care system suggests that vaccines remain highly effective against severe COVID-19, even in the age of the delta variant. However, we found evidence of waning protection after 200 days, especially in those older than 80 years or with certain clinical comorbidities. Additional protection (eg, a booster vaccination) might be warranted for everyone, but especially for these populations. In addition to promoting general vaccine uptake, clinicians and policy makers should consider prioritising booster vaccinations in those most at risk of severe COVID-19.

Data sharing

Upon request, beginning 6 months after publication and ending 5 years after publication, the authors will share fully de-identified individual participant data that underlie the results reported in this Article upon request for additional analysis of vaccine effectiveness. Data will be considered for sharing with investigators whose use of the data has been approved by an independent institutional review board. The corresponding author should be contacted for access.

Declaration of interests

We declare no competing interests.