Relative effectiveness of booster vs. 2-dose mRNA Covid-19 vaccination in the Veterans Health Administration: Self-controlled risk interval analysis.

OBJECTIVE: To estimate relative effectiveness of the booster mRNA Covid-19 vaccination versus the 2-dose primary series for both Delta and Omicron variants with self-controlled study design.
METHODS: We used the Veterans Health Administration (VHA) Corporate Data Warehouse to identify U.S. Veterans who received the 2-dose primary mRNA Covid-19 vaccine series and a mRNA Covid-19 booster, and who had a positive SARS-CoV-2 test during the Delta (9/23/2021-11/30/2021) or Omicron (1/1/22-3/19/22) predominant period. Among them, we conducted a self-controlled risk interval (SCRI) analysis to compare odds of SARS-CoV-2 infection during a booster exposure interval versus a control interval. Exposures were a control interval (days 4-6 post-booster vaccination, presumably prior to gain of booster immunity), and booster exposure interval (days 14-16 post-booster vaccination, presumably following gain of booster immunity). Cases had a positive PCR or antigen SARS-CoV-2 test. Separately for Delta and Omicron periods, we used conditional logistic regression to calculate odds ratios (OR) of a positive test for the booster versus control interval and calculated relative effectiveness of booster versus 2-dose primary series as (1-OR)*100. The SCRI approach implicitly controlled for time-fixed confounders.
RESULTS: We found 42 individuals with a positive SARS-CoV-2 test in the control interval and 14 in the booster exposure interval during the Delta period, and 141 and 70, respectively, in the Omicron period. For the booster versus 2-dose primary series, the odds of infection were 70% (95 %CI: 42%, 84%) lower during the Delta period and 54% (95 %CI: 38%, 66%) lower during Omicron. In sensitivity analyses among those with prior Covid-19 history, and age stratification, ORs were similar to the main analysis.
CONCLUSIONS: Booster vaccination was more effective relative to a 2-dose primary series during the Delta and Omicron predominant periods, and the relative effectiveness was consistent across age groups.

Introduction

Following the Food and Drug Administration (FDA) Emergency Use Authorization (EUA) for the mRNA Covid-19 vaccines, several studies demonstrated high vaccine effectiveness (VE) for these vaccines in real-world settings in the United States (US).[1], [2], [3], [4], [5] By July 2021, Delta became the predominant circulating SARS-CoV-2 variant in the US, and reports of breakthrough infections rose along with questions regarding waning immunity of the mRNA vaccines.[6], [7] These occurrences prompted the Advisory Committee on Immunization Practices to recommend booster vaccination and, starting in September 2021, to allow a booster dose for a larger portion of the population, including those ≥ 65 years old or any adult with an underlying medical condition or increased risk of exposure to Covid-19.[8], [9], [10].

Booster vaccine effectiveness (VE) for mRNA vaccines from real world settings in and outside the US have shown lower VE against infection with Omicron than Delta variants. A study in southern California for December 2021 reported lower booster vaccine effectiveness (VE) against infection for Omicron (62.5%) than Delta (95.2%).[11] A CDC-led study across the US reported booster VE against infection declined from 93% to 80% from the period of Delta predominance to Omicron emergence[12], and a separate CDC-led study found similar declines of 94% to 82% among patients tested in emergency department/urgent care encounters.[13] In Israel the recommendation for booster vaccination preceded the US;[14] analyses of patient data from August-September 2021, when Delta was predominant, showed risk of infection was 12 times lower among those boosted versus those not boosted.[14], [15].

Since observational studies contribute to our understanding of VE, and policy decisions and scientific recommendations are based in part upon observational studies, it is important to have confidence in the findings. Observational studies from different settings and periods of the predominant circulating SARS-CoV-2 variant contribute to our understanding of VE. All observational studies must account for confounding, and studies of VE must account for differences between vaccinated and unvaccinated people that may contribute to different risk of infection. Observational studies must also minimize misclassification of vaccination status which can bias results. To reduce such confounding, we utilized a self-controlled study design that implicitly accounts for time-fixed confounders. Using data from veterans who were recorded as fully vaccinated with two doses of mRNA Covid-19 vaccines and later boosted, we quantified odds of infection for booster vaccination versus the 2-dose primary series. We used data from the Veterans Health Administration (VHA) population, which offers a unique opportunity to better our understanding of the booster vaccination across the US given the large number of individuals who were vaccinated and later boosted at the VHA and who also utilize testing at VHA facilities. Because of the ability to analyze data in near real-time, it was possible to assess the effectiveness of the booster with the establishment of Omicron as the predominant variant.

Methods

Data sources

The VHA is the largest integrated health care system in the U.S., providing comprehensive care to over nine million veterans at>171 medical centers and 1,112 outpatient sites of care.[16] Electronic medical record data from the VHA Corporate Data Warehouse (CDW) were analyzed. We used publicly available data from the Centers for Disease Control and Prevention on weekly monitoring of variant proportions in the US to identify the predominant SARS-CoV-2 variants for time periods in the study.[17].

Study design

We used a self-controlled risk interval (SCRI) study design, a variation of the self-controlled case series (SCCS) design.[18] The method can be used for non-recurrent events when the risk of occurrence over the study period is 10% or less, [19] which was the case with the current study. With this design, only cases are included in the analysis, and periods of exposure and non-exposure around an event of interest are identified. The SCRI design is beneficial when studying an exposure where identifying an unbiased comparable cohort is difficult, as in our case in which people with booster vaccination are potentially different from the non-boosted in multiple measurable and unmeasurable ways, thus increasing the risk of residual confounding for cohort or case-control designs despite adjustment. In SCRI, time-fixed confounders are implicitly adjusted for because the risk and control intervals belong to the same individual and, accordingly, the analysis is matched. We shortened the length of intervals for analysis to be segments of the exposure and non-exposure periods to decrease time-varying confounding as well. Furthermore, by restricting the analysis only to patients with recorded booster vaccination we avoided misclassification of booster vaccination status that may occur in alternate study designs where an unvaccinated group is needed and vaccinations received outside of a healthcare system can lead to misclassification.

A prior study by Bar-On et al [15] of the BNT162b2 messenger mRNA vaccine, assumed that anamnestic immune response would start at around day seven post-booster vaccination and that testing is likely to follow infection by 5 days (incubation period). They [15] selected days > 12 post-booster vaccination as the time period during which the vaccinee should have already benefited from the booster dose. Following a similar logic, we used 4–6 days post-vaccination as the “control interval” to represent non-boosted exposure status, assuming a positive SARS-CoV-2 test during this time would likely reflect infection that occurred prior to the effect of booster vaccination. Also a priori we selected a “booster exposure” interval of days 14–16 post-booster vaccination, of the same length (3 days) as the control interval, specifically to represent a short interval within the presumed boosted effect time period that was also close (within a two-week period) of the control interval, minimizing the potential for large differences in community transmission of SARS-CoV-2 between the booster exposure and control intervals (Fig. 1 ). Minimizing difference in community transmission between the intervals is important so that the likelihood of infection due to factors other than vaccination is similar for the booster and control intervals. The control and booster exposure intervals from the same individual formed a matched pair for analysis.

Fig. 1

Booster exposure and control intervals for analysis.

Study population

This analysis focused on VHA-enrolled veterans who were vaccinated and later boosted with an mRNA Covid-19 vaccine. The study population was restricted to veterans who received two Pfizer-BioNTech Covid-19 vaccines or two Moderna Covid-19 vaccines during December 14, 2020-August 1, 2021 and subsequently received a third mRNA vaccine (i.e., booster) of either mRNA vaccine type September 23, 2021 or thereafter. We restricted the population to vaccinees who received their 2-dose primary series by August 1, 2021 because we were interested in examining the booster effect during Delta and Omicron predominant periods through March 19, 2022, and most individuals who received a 2-dose primary series after August 1, 2021 would not be recommended for booster vaccination given the recency of their 2-dose primary series. Vaccinees were veterans living in the US, enrolled in VHA ≥ 2 years prior to the vaccination era (December 14, 2020), and had ≥ 1 visit to a VHA facility in the prior 2 years. In the main analysis, we did not include those individuals who had a prior Covid-19 diagnosis, had a positive antigen test or a positive PCR test prior to booster vaccination. As a condition of the SCRI study design, only vaccine recipients who had a positive PCR or antigen SARS-CoV-2 PCR test during the control interval (i.e., days 4–6 post booster vaccination) or booster exposure interval (i.e., days 14–16 post booster vaccination) were included in the analysis. Tests from veterans who were hospitalized for more than one day at time of testing were excluded.

We classified SARS-CoV-2 tests during September 23, 2021- November 30, 2021 as being from the Delta-predominant period and those in January 1-March 19, 2022 from the Omicron-predominant period, based on the CDC’s data which showed nearly 100% of sequenced samples were Delta during August 2021-November 2021, and 90–100% were Omicron January 1-March 19, 2022.[17] We excluded tests taken from veterans during December 2021 as the predominant variant shifted during this time. Tests had to be from individuals whose booster exposure interval and control interval fell within the variant-predominant period of analysis.

Exposure, outcome and covariate assessment

Using the positive SARS-CoV-2 test data from the population of vaccinees who met the study criteria, we classified individuals who had a positive test in the booster exposure or control intervals as cases and determined the exposure status of the case by whether the test date occurred in the booster exposure or control interval.

The SCRI design adjusts for confounding by matching an individual to themself close in time, minimizing or eliminating the need for additional adjustment. We did adjust for the booster exposure interval or control interval including a weekend day, as an individual’s likelihood of being tested could be different during a weekend. We included other variables in a descriptive analysis and used them to conduct stratified analyses.

We conducted a negative control exposure analysis using the exposure of influenza vaccination (not on the same day as Covid-19 vaccination) at a VHA facility and compared risk of a positive SARS-CoV-2 test days 14–16 versus days 4–6 post influenza vaccination as we would expect that the risk of SARS-CoV-2 infection to be no different for these time periods around influenza vaccination. With negative control exposure analysis, an exposure which has no effect on the outcome of interest but is subject to the same confounding as the study exposure is chosen, so any association between the negative control and the outcome would indicate uncontrolled confounding in the study.[20].

Statistical analysis

The population of vaccinees who met the study criteria for the Delta and Omicron periods were identified and described. Mean (standard deviation) and median (interquartile range) were reported for continuous variables, and frequency and proportions reported for categorical variables. In this SCRI analysis, we used conditional logistic regression to calculate the odds ratio (OR) (95% confidence interval [CI]) of a positive test for the booster exposure versus the control interval, and conducted analyses separately for the Delta predominant and Omicron predominant periods and for subgroups stratified by age (<65 years and ≥ 65 years). We estimated the relative effectiveness of the booster versus the 2-dose primary series as the percentage reduction in the odds of testing positive for the low-risk booster exposure interval versus the control interval ([1-OR]*100%).

Sensitivity analyses

Additional analyses were conducted including patients with a history of Covid-19 prior to booster vaccination so long as the last SARS-CoV-2 positive test was ≥ 90 days prior to the SARS-CoV-2 test following booster vaccination.

All analyses were conducted using SAS, version 9.4.

Approval: The study protocol was approved by the institutional review board of the VA Medical Center in White River Junction, VT and was granted an exemption of consent.

Results

A total of 2,651,113 Veterans had completed the 2-dose primary series by August 1, 2021 and had not been boosted by September 23, 2021. Among them, 1,353,944 (51%) received a booster September 23, 2021- March 19, 2022. The median time from 2nd dose until booster was 247 days (range 59–437 IQR 227–271). Time between 2nd dose and booster was shorter for those boosted in the Delta versus Omicron predominant periods (Delta: median 237 days (IQR 220–254), Omicron: median 297 days (IQR 272–323). Among those boosted in this time interval, 10,078 (0.74%) had a positive SARS-CoV-2 test after the booster, with 309 during the Delta period, 8,260 during the Omicron period, and the remaining in December 2021. Table 1 displays characteristics of individuals who had a positive SARS-CoV-2 test after their mRNA Covid-19 booster by variant predominant period. Among those with a positive test in the Delta and Omicron predominant period, there were 97 (31%) and 2,141 (26%) with an immunocompromising condition, respectively.

Table 1

Characteristics of individuals with positive SARS-CoV-2 test following booster vaccination during Delta and Omicron periods.

	Delta period (n = 309)	Omicron period (n = 8,260)
Age, years, median (IQR)	72 (66, 77)	69 (59, 74)
Sex, n(%)
Male	286 (93)	7,511 (9 1)
Female	23 (7)	749 (9)
Race/ethnicity, n(%)
Non-Hispanic white	208 (67)	4,663 (56)
Non-Hispanic black	45 (15)	2,194 (27)
Hispanic any race	17 (5)	432 (5)
Other/ unknown	39 (13)	971 (12)
Rurality, n(%)
Rural	96 (31)	1,922 (2 3 )
Urban	210 (68)	6,298(76)
Missing	3 (<1)	40 (<1)
Booster manufacturer, n(%)
Pfizer	163 (53)	4,152 (50)
Moderna	146 (47)	4,108 (50)
2-dose primary series manufacturer, n(%)
Pfizer	167 (54)	4,122 (50)
Moderna	142 (46)	4,138 (50)
Comorbidities1, n(%)
Asthma	55 (18)	1,433 (17)
Cancer	47 (15)	982 (12)
Chronic kidney disease	51 (16)	1,123 (14)
COPD	56 (18)	1,539 (19)
Cardiovascular disease	34 (11)	583 (7)
Diabetes, with complications	68 (22)	1,431 (17)
Diabetes, without complications	129 (42)	3,021 (37)
Hypertension	213 (69)	5,361 (65)
Immunocompromised	97 (31)	2,141 (26)
Obesity	60 (19)	1,556 (19)

Abbreviations: COPD, chronic obstructive pulmonary disease; IQR, interquartile range.

See Supplemental Table 1 of Young-Xu et al.[25] for definitions of variables in this table.

We restricted the analysis population to individuals with a SARS-CoV-2 positive test during the control or booster exposure intervals which included 56 individuals during the Delta period and 211 during the Omicron period. In the Delta period, there were 42 cases in the control interval, and 14 in the booster exposure interval. In the Omicron period, there were 141 cases in the control interval, and 70 in the booster exposure interval. The characteristics of the study population are shown in Table 2 . There were 21 (38%) and 45 (21%) with an immunocompromising condition in the Delta and Omicron predominant periods, respectively. Among those infected during the Delta predominant period, the median number of days from second to booster dose was 234 (IQR 217–254), and for those infected during the Omicron predominant period the median was 287 (IQR 259–310).

Table 2

Characteristics of individuals with positive SARS-CoV-2 test during Delta and Omicron periods included in SCRI analysis.

	Delta period (n = 56)	Omicron period (n = 211)
Age, years, median (IQR)	72 (67, 75.5)	60 (49, 72)
Sex, n(%)	*
Male		184 (8 7)
Female		27 (1 3)
Race/ethnicity, n(%)	*
Non-Hispanic white		100 (47)
Non-Hispanic black		67 (32)
Hispanic any race		18 (9)
Other/ unknown		26 (12)
Rurality, n(%)
Rural	21 (38)	54 (2 6)
Urban	35 (62)	157 (7 4)
Booster manufacturer, n(%)
Pfizer	31 (55)	79 (37)
Moderna	25 (45)	132 (63)
2-dose primary series manufacturer, n(%)
Pfizer	31 (55)	80 (38)
Moderna	25 (45)	131 (6 2)
Comorbidities1, n(%)
Asthma	*	25 (12)
Cancer	11 (20)	17 (8)
Chronic kidney disease	*	25 (12)
COPD	*	26 (12)
Cardiovascular disease	*	13 (6)
Diabetes, with complications	14 (25)	30 (14)
Diabetes, without complications	22 (39)	61 (29)
Hypertension	39 (70)	121 (57)
Immunocompromised	21 (38)	45 (21)
Obesity	14 (25)	50 (24)

Abbreviations: COPD, chronic obstructive pulmonary disease; IQR, interquartile range.

Reporting of number suppressed due to low value because of privacy concerns.

See Supplemental Table 1 of Young-Xu et al.[25] for definitions of variables in this table.

We presented the estimated relative effectiveness of the booster versus the 2-dose primary series in Table 3 . There was a 70% (95 %CI: 42%, 84%) reduction in the odds of testing positive in the booster exposure versus control interval during the Delta period. There was a reduction in the odds of testing positive for the booster exposure versus control interval during the Omicron period as well (54% (95 %CI: 38%, 66%)). The estimated relative effectiveness of the booster versus 2-dose primary series was similar for individuals < 65 years and ≥ 65 years during the Omicron period. We did not conduct stratified analysis for the Delta period given small numbers.

Table 3

Relative effectiveness of booster vaccination versus 2-dose primary series.

	Number of positive SARS-CoV-2 tests
	Booster exposure interval (days 14–16 post booster vaccination)	Control interval (days 4–6 post booster vaccination)	Relative effectiveness (95% CI) [reference: control interval]1
Delta period	14	42	70% (42%, 84%)
Omicron period	70	141	54% (38%, 66%)
<65 years old	39	84	59% (37%, 73%)
≥65 years old	31	57	48% (18%, 67%)

Delta period: September 23, 2021-November 30, 2021; Omicron period: January 1, 2022- March 19, 2022.

Relative effectiveness is the percentage reduction in the odds of testing positive for the booster exposure versus the control interval = ([1-ORodds of SARS-CoV-2 in booster exposure interval vs odds of SARS-CoV-2 in control interval]*100%).

In the negative control exposure analysis we found no statistically significant association between the time interval around influenza vaccination and SARS-CoV-2 infection (OR of SARS-CoV-2 infection for days 14–16 vs days 4–6 post influenza vaccination: [OR = 1.38 (95%: 0.89, 2.2)]).

For the sensitivity analysis, including veterans with a prior history of Covid-19 added few observations to the study population: 60 had positive tests (cases) during the Delta period (43 in the control interval, 17 in the booster exposure interval) and 225 during the Omicron period (153 cases in the control interval, 72 cases in the booster exposure interval). The estimated reduction in odds of infection for booster exposure versus control interval was similar to the main analysis (Delta: 66% (95 %CI: 37%, 82%); Omicron: 56% (95 %CI: 41%, 68%)).

We conducted a post hoc analysis for both variant predominant periods where the analysis was stratified at 240 days between second dose and booster dose given the difference in number of days between second dose and booster dose for those infected during the Delta predominant period and Omicron predominant period. There was a 70% (95%CI: 27%, 87%) reduction in the odds of testing positive in the booster exposure versus control interval during the Delta period for those with ≤ 240 days between second and booster dose, and a 71% (95 %CI: 20%, 89%) reduction for those with > 240 days. For the Omicron predominant period the reduction was 65% (95 %CI: 15%, 85%) and 54% (95 %CI: 36%, 67%) for ≤ 240 days and > 240 days, respectively, between second dose and booster dose.

Discussion

In this study, the booster vaccination was associated with a 70% reduction in infection compared with the 2-dose primary series during the Delta period and 54% reduction during Omicron. These differences were similar for those ages < 65 and ≥ 65 years. Findings from this study align with prior observational studies showing that booster vaccination is associated with lower odds of infection compared with the 2-dose primary series.[11-1521-23] Prior studies [11-1322] showed greater estimated protection of booster vaccination versus the 2-dose primary series during the Delta compared with the Omicron period; while the point estimates from our analysis were consistent with this, the difference in protection between the two periods was not conclusive due to overlap in wide confidence intervals.

While most other studies evaluating booster vaccination relied on comparison with unvaccinated individuals, a limited number of studies [15], [21], [23] have compared risk of infection for booster vaccination versus the 2-dose primary series (although not with a SCRI approach). One study conducted among individuals boosted in Israel during the Delta predominant period reported risk of infection was five times lower ≥12 days after the booster versus days 4–6 post-booster.[15] A case-control study conducted in Israel during the Delta wave compared odds of infection for individuals with booster versus the 2-dose primary series, estimating an 83–87% reduction in risk for booster vaccination;[23] this is comparable to the 70% relative effectiveness (OR = 0.30) that we report for booster versus 2-dose primary for the Delta period. In a study of public health testing sites in the US, Accorsi et al [22] estimated relative effectiveness was 84% for 3 doses versus 2 doses for confirmed Delta infection and 66% for Omicron infection, comparable to the 54% we report for the Omicron period. The reduction in odds of infection for 3 doses versus 2 is incremental, that is, on top of the reduction in odds gained from 2 doses compared to unvaccinated.

The main strengths of this study are that by using a SCRI design we implicitly adjusted for all time-fixed confounders, and the very short interval we chose between windows minimizes the likelihood of all time varying confounding, including by differences in virus circulation. Also, our near real-time access to medical records allowed analysis of the booster dose for the Omicron period. Misclassification of vaccination status in other studies is often a concern as individuals may be vaccinated and boosted outside of their regular places of care and vaccination records may not be updated to reflect that for some time. We eliminated that risk by designing the study to only include those with known vaccination. While everyone in this study had three vaccinations at the VA, it is possible that some may have obtained additional vaccinations elsewhere. However, our population was limited to veterans who routinely sought care at VHA facilities and given they had three vaccinations at a VHA facility they were unlikely to have also been vaccinated elsewhere.

We included days 4–6 post-vaccination as the control window, assuming immune response to vaccination will not occur so early (moreover, testing date is often subsequent to symptoms onset day); if some immune response to the booster dose was present during this control period, the bias would have been towards the null. Thus, our estimates of relative effectiveness of booster versus the 2-dose primary series may even be conservative. While the SCRI study design adjusts well for person-level factors that remain constant across exposure intervals, it is possible that individuals’ behaviors changed around time of vaccination which could alter infection risk. If individuals were more likely to reduce masking and other distancing measures following booster vaccination this could have underestimated the effect of booster vaccination during the booster exposure period when compared to the control period. Because the Delta and Omicron analysis periods occurred several months after the recommendation for three dose vaccination for individuals with immunocompromising conditions, we assume that our analysis is most relevant to individuals without immunocompromising conditions as those with the conditions would have received a third dose earlier.

Because we did not differentiate between symptomatic and asymptomatic disease, we did not measure effectiveness specifically for prevention of symptomatic COVID-19 disease. Also, we did not consider the time interval between 2nd dose and booster dose in our main analysis; thus, we did not account for waning of effectiveness of the second dose. Therefore, our results show the relative effectiveness between a booster dose and 2 doses at the actual time at which the veterans received their booster dose (among veterans in this study, 247 days on average between 2nd dose and booster vaccination). Waning of effectiveness has been studied elsewhere, and one study reported VE against infection during the Delta period was 86% versus 76% for the 2-dose series when the time since second dose was < 180 versus ≥ 180 days; the decrease in VE by time since vaccination during Omicron was similar (52% to 38%).[13] Booster vaccination among individuals with such a long time since their 2nd dose could potentially have greater benefit from booster vaccination relative to those boosted closer in time to their 2nd dose. While our post hoc analysis explored this, the confidence intervals for relative effectiveness were wide after stratifying by time between second dose and booster dose. Also, our study only assessed the booster effect shortly after vaccination and, thus, did not address potential waning of protective immunity following the booster vaccination. Viral sequencing data were not available for each SARS-CoV-2 lab test included in this study, so we relied on CDC variant tracking data to classify cases as occurring during the Delta and Omicron predominant periods. To increase confidence in regard to the variant, we excluded the month of December 2021, during which the Delta-Omicron dominance was changing. Given that CDC data indicated near 100% Delta and Omicron predominance in our designated variant predominant periods, this was a reasonable assumption. While the VHA population is predominantly male and is not generalizable in every way to the general US population, a majority of older male VHA enrollees are similar to Medicare beneficiaries in terms of demographic and health characteristics, suggesting a high degree of generalizability.[24].

Conclusion

This observational study indicates mRNA vaccine boosters were associated with a significant reduction in risk of infection relative to the 2-dose primary series during Omicron predominance, and during the Delta predominant period.

Declaration of Competing Interest

CK, JS, GZ, EP and YY-X acknowledge having received funding form Pfizer for other research projects ohter than this one. HI and KMR report no competing interests.