Reliably Assessing Duration of Protection for COVID-19 Vaccines.

Decision-making about vaccination and boosting schedules for COVID-19 hinges on reliable methods for evaluating the longevity of vaccine protection. We show that modeling of protection as a piecewise linear function of time since vaccination for the log hazard ratio of the vaccine effect provides more reliable estimates of vaccine effectiveness at the end of an observation period and also more reliably detects plateaus in protective effectiveness as compared with the traditional method of estimating a constant vaccine effect over each time period. This approach will be useful for analyzing data pertaining to COVID-19 vaccines and other vaccines where rapid and reliable understanding of vaccine effectiveness over time is desired.

The coronavirus disease 2019 (COVID-19) pandemic has been mitigated by the deployment of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) vaccines, together with the adoption of standard public health measures. However, the protective effects of vaccines wane over time [1-7]. Understanding the exact nature of waning vaccine efficacy or vaccine effectiveness (VE), particularly against severe disease, would enable properly informed decision making about the need for and the optimal timing of booster shots [8]. Likewise, it is important to evaluate the duration of protection afforded by booster shots and the need for further boosting. In addition, there is an urgent need for universal coronavirus or pan-sarbecovirus vaccines, and the durability of protection afforded by such new vaccines will also need to be investigated [9]. Here, we discuss the sensitivity of existing analyses to waning VE and show how changes in disease risks over calendar time can influence that sensitivity. We then present an approach with greater sensitivity that can properly account for other influencing factors, such as emerging viral variants. Improved assessments of real-world VE using this approach could also have major implications for decision making regarding other products.

Waning protection is commonly assessed by comparing VE estimates over successive time periods [1-6]. The VE estimates are obtained under the standard Cox or Poisson model, assuming a constant VE over each time period, and thus will be referred to as “VEConst” hereafter. In the presence of waning, VEConst represents a weighted average of the time-varying vaccine effects over the time period, weighted by when the events occur (ie, the vaccine effects at the time points with more events receive greater weights), and thus tends to be higher than the true VE at the end of the time period, especially when that period is long. In addition, VEConst tends to be less precise when each time period is short, such that larger studies are required to draw firm conclusions.

To obtain more precise and up-to-date estimates of protection, we advocate fitting a Cox model with 2 time indexes: the event times are measured from the start of the study in calendar time, and the log hazard ratio for the vaccine effect is a continuous, piecewise-linear function of time elapsed since vaccination [7, 10, 11]. The corresponding VE on the hazard rate (VEHR) represents piecewise exponential deterioration of vaccine effect by time since vaccination [11, 12]. Because it measures the vaccine effect on the instantaneous risk of disease at the current time, rather than an overall benefit over a broad time period, VEHR is more sensitive to the level of waning than VEConst. In addition, measuring time to disease occurrence from trial initiation allows us to account for waxing and waning infection rates and compare disease incidence between the vaccinated and unvaccinated groups at the same calendar time [7, 10, 11, 13, 14].

To illustrate the relative sensitivity of these approaches, we simulated a clinical trial mimicking the enrollment pattern of the BNT162b2 study [1] and the trend of COVID-19 infections occurring in the United States during that trial (Supplementary Materials). We assumed that the true VEHR of a hypothetical vaccine decreases (linearly in the log hazard ratio) from a peak of 95% at full vaccination (ie, 7 days after dose 2) that lasts 1 month to 70% at 6 months after full vaccination [10, 11]. The means of VEConst over 1000 replicates are 94.4%, 89.9%, and 81.6% over 0–2, 2–4, and 4–6 months, respectively. The degree of waning is overestimated by VEConst at the beginning of each time period and is underestimated at the end of that period (Figure 1). The underestimation by VEConst of the true level of waning was accentuated, even when estimation is performed within 2-month intervals, because vaccinations tended to coincide with an early peak in the incidence of infections, and then this incidence rate waned for many months thereafter. This resulted in a high percentage of exposures occurring during the earlier part of each 2-month interval when the true VE was higher.

Figure 1.

Estimation of vaccine efficacy (VE) against symptomatic coronavirus disease 2019 ( COVID-19) based on 6 months of follow-up in 4 simulated clinical trials. In the first 2 trials, the true vaccine efficacy on the hazard rate (VEHR) (“truth”) decreases (linearly in the log hazard ratio) from a peak of 95% at full vaccination that lasts 1 month to 70% at 6 months after full vaccination. In the trial depicted in A, most participants received dose 2 at a calendar time coinciding with a peak in infection rates, whereas in the trial depicted in B, most participants received dose 2 at a time of low infection rates. In the trials depicted in C and D, the true VEHR plateaus at 5 and 3.5 months, respectively. In each trial, VEConst (VE estimate obtained under the standard Cox or Poisson model, assuming a constant VE over each time period), is obtained over 0–2, 2–4, and 4–6 months after full vaccination, and VEHR is estimated under the Cox model, in which the log hazard ratio is a piecewise linear function of time since vaccination, with change points at 0, 2, and 4 months after full vaccination. For each trial, the mean and standard deviation of each estimator over 1000 replicates are shown by the solid curve and shaded area, respectively.

We simulated a second trial by shifting the enrollment period to 6 months later, such that the period with the strongest vaccine effects coincided with a nadir in exposure rates (Supplementary Materials). Then the means of VEConst over 1000 replicates are 94.7%, 89.5%, and 78.8% over 0–2, 2–4, and 4–6 months, respectively (Figure 1). VEConst, in essence providing estimates of VE at the mid-points of these 2-month intervals, does not have the same level of overestimation of VE in the second trial relative to the first, in that the mean of VEConst over 4–6 months is closer to the true VE at month 5 in the second trial than in the first trial. In both trials, the estimated VEHR curve is close to the truth (Figure 1 and 1).

Neutralizing antibodies conferring short-term protection could wane log-linearly, leading to waning of VE over several months, yet for a lengthy duration thereafter, VE could be maintained at a plateau owing to cell-mediated or memory immune responses that remain nearly constant over time. Thus, we simulated 2 more trials by letting the true VE reach a plateau at 5 months after full vaccination in the first trial and at 3.5 months in the second (Supplementary Materials). In the first trial, 6-month VE is somewhat overestimated by VEConst and somewhat underestimated by VEHR (Figure 1). In the second trial, both VEConst and VEHR provide estimates of 6-month VE that are close to the truth (Figure 1). Importantly, the information obtained from VEHR allows more rapid detection of nonlinear changes (such as this plateau) in VE over time, while analysis using VEConst could detect a plateau only with a longer follow-up period.

Owing to the crossover of placebo recipients to the vaccine arm, phase 3 trials have provided efficacy information only for approximately 6 months after dose 2 [1, 2], although it is possible to recover placebo-controlled VE approximately 6 months after crossover under certain assumptions [10, 13, 14]. Observational studies can provide information about longer-term VE. Moreover, large observational databases enable estimation of VE against severe disease and against different viral strains, as well as in various subpopulations. The aforementioned VEHR curve provides similar advantages over VEConst in assessment of waning VE in the observational setting. These advantages are apparent when contrasting the VEConst estimates reported in a British study versus the VEHR curves reported in a US study [6, 7].

The reduction in VE over calendar time or as the time since vaccination increases may be caused by decline of immunity to the primary vaccination, by emergence of new variants that evade antibody recognition, or by both. Comparing VE at a given calendar time among individuals who were vaccinated at different dates allows assessment of waning VE due to declining immunity, and comparing VE at different calendar times for individuals who have been vaccinated for the same amount of time allows assessment of waning VE due to new variants. Such comparisons revealed that the increase in postvaccination SARS-CoV-2 infections during the summer and early fall of 2021 was due to both declining vaccine-induced protection and the emergence of the delta variant [7].

The effectiveness of a boosting program could be evaluated ideally by large-scale randomization and practically by observational studies. The effect of the booster shot on the hazard rate of disease as a function of time since booster vaccination could be incorporated into the time-varying hazard ratio in the Cox model [7, 10, 11]. The corresponding VEHR curve would provide useful insights into whether and when further boosting is needed.

The proposed approach based on VEHR improves sensitivity for evaluating the true durability of VE using data from phase 3 clinical trials and observational studies, by allowing VE to vary continuously by time after vaccination and by adjusting for changes in disease incidence over calendar time. To reduce confounding bias, analyses of observational data should adjust for individual characteristics (eg, priority tier, age, sex, and race/ethnicity) and the influences of calendar time and geographical location, as well as other factors (eg, emerging viral variants or vaccination prioritization programs that allow high risk cohorts to be vaccinated first). It is important to recognize that additional factors not easily addressed through modeling (eg, having a declining number of “controls” over calendar time who remain truly unvaccinated, given an increasing percentage having had infections or vaccinations obtained outside primary healthcare systems) could be influential. Therefore, changes in underlying immunization rates, disease incidence among vaccinees and controls, and follow-up rates among vaccinees and controls over relevant time periods should also be reported. As the population becomes antigen exposed, we will only be able to study vaccine durability relative to the current level of population immunity.

There is increasing interest in using postmarketing VE data to support regulatory and deployment decisions for other vaccines. For example, observational studies supported the recent Canadian approval of diphtheria, tetanus, acellular pertussis vaccine administered to pregnant women, and long-term observational data supporting effectiveness of a zoster vaccine was included in a US package insert. Rapid and reliable analysis of observational data using the methods proposed here would be useful in confirming the effectiveness of vaccines approved under US accelerated approval, European Medicines Agency conditional approval, or other analogous mechanisms.

Click here for additional data file.