Intention-to-prevent analyses for estimating human papillomavirus vaccine efficacy in clinical studies.

HPV vaccine efficacy trials have been conducted in populations exposed to HPV infection (i.e., sexually active individuals); participants were not excluded from participating in the trials based on their HPV status at baseline. Thus, some participants could have been infected at baseline with 1 or more vaccine HPV types. Because HPV vaccines are prophylactic and do not affect existing HPV infections, prophylactic efficacy was assessed in a per-protocol population (those not infected at enrollment to the HPV type being analyzed who also completed the 3-dose regimen of vaccine and had no protocol violations). Supportive intention-to-treat (ITT) and modified ITT, were also conducted to include those with prevalent HPV infection. ITT analyses included those who received ≥1 dose of vaccine and had efficacy follow-up regardless of whether or not they were infected with HPV prior to vaccination. Efficacy in the ITT population simply reflects the amount of prevalent infection in a particular population of study subjects. Intention-to-prevent (ITP) analyses included those who received one dose of vaccine, had efficacy follow-up, and were not infected at enrollment to the HPV type being analyzed. While all of these analyses have been presented, there has been little discussion regarding their respective significance. In this methodological review, we show that an ITT analysis does not preserve an unbiased comparison of treatment groups in relation to estimating prophylactic HPV vaccine efficacy. Furthermore, ITP is more suitable at preserving an unbiased comparison of treatment groups in relation to estimating prophylactic HPV vaccine efficacy.

Introduction

Human papillomavirus (HPV) causes nearly all cervical cancer cases, as well as substantial proportions of anal, vulvar, vaginal, penile and oropharyngeal cancers [1]. The licensed quadrivalent HPV 6/11/16/18 (4vHPV) vaccine and bivalent HPV16/18 (2vHPV) vaccine address oncogenic HPV types 16 and 18 that cause approximately 70% of cervical cancer cases worldwide [2]. The licensed nine-valent HPV (9vHPV) vaccine addresses the oncogenic HPV types 16/18/31/33/45/52/58 which cause approximately 90% of cervical cancer cases worldwide [3], [4], [5].

The clinical trials evaluated HPV vaccine efficacy by using pre-cancerous lesions as the primary efficacy surrogate endpoints for invasive cervical cancer. Such clinical trials were conducted on sexually active individuals 16–26 years of age who were at-risk for becoming infected with HPV and developing pre-cancerous cervical lesions. The time from acquisition of infection to the development of precancerous lesions (e.g., cervical intraepithelial neoplasia grade 2 or worse, which is the obligate precursor of cervical cancer) can take up to 90 months [6]. Moreover, the standard of care is to screen for and excise precancerous lesions to prevent invasion.

The licensed prophylactic HPV vaccines consist of virus-like particles (VLPs) composed of the viral capsid protein L1 of each HPV type in the vaccines. These vaccines were expected and confirmed to be strictly prophylactic in nature [5], [7], [8], [9], [10]. To demonstrate prophylactic HPV vaccine efficacy, the definitive clinical trials could have screened and recruited only women who were uninfected at baseline; however, this approach would have produced a highly selected population with unknown biases that would not represent the population in which the vaccine would be subsequently used. Additionally, if only individuals without HPV infection were eligible for the pivotal efficacy trials, HPV vaccines would likely be indicated only in individuals who are HPV-negative, which would make vaccination programs infeasible for sexually active individuals.

Approximately 60% of sexually active persons will become infected with HPV during their lifetime, thus many enrolled in HPV vaccine efficacy clinical trials could already be infected with one or more of the HPV types that the vaccine is designed to protect against, or with HPV types that are not in the vaccines [11], [12]. However, infection with all HPV types in each of the 3 HPV vaccines is rare. For instance, infection with all four types present in the 4vHPV vaccine is 0.1% of 3578 women enrolled in North America by serology or HPV DNA [13], and none were infected with all nine HPV types in 9vHPV vaccine, so essentially all subjects vaccinated with 2vHPV, 4vHPV or 9vHPV vaccines would potentially derive some benefit by being vaccinated [14], [15].

Intention-to-treat analysis is frequently viewed as a mainstay of unbiased analysis of randomized clinical trials because its basic premise is to preserve the unbiased comparability of treatment groups with respect to estimation and subsequent inference relating to treatment effect. It is commonly accepted that sub-dividing study populations into analysis cohorts benefits the modality under study, and that an ITT analysis provides a more valid estimate of overall efficacy. In the context of HPV vaccination, however, estimation of overall efficacy via an ITT approach without critical assessment of what ‘overall’ means leads to inappropriate conclusions because such prophylactic vaccines make no claim for a therapeutic effect and, in fact, have demonstrated no therapeutic effect [5], [7], [8], [9], [10].

This article discusses the limitations of ITT analyses in the context of efficacy trials of prophylactic HPV vaccines and proposes that an alternative intention-to-prevent (ITP) analysis should be preferred.

Methods and results

Limitations of ITT analyses in HPV vaccine clinical trials

Assessment of vaccine efficacy in different analysis populations

By way of explanation, the metric called vaccine efficacy is a percent risk reduction, calculated as 100% x (1 minus the relative risk). In HPV vaccine clinical trials, relative risk is typically calculated as the risk of disease in the “innovator vaccine group” divided by the risk of disease in the “control group”. The control group can be a placebo group or an existing standard-of-care vaccine. Risk of disease can be an incidence rate, or count of disease cases if the innovator vaccine group and the control group have approximately equal follow-up times. Thus, the vaccine efficacy metric is the percent reduction in the control group risk of disease that the innovator vaccine can generate.

In HPV vaccine clinical trials, vaccine efficacy against HPV types covered by a particular HPV vaccine was evaluated on an HPV-type-specific manner and conducted by identifying individuals in the sexually active efficacy population who are not infected at baseline with the vaccine-HPV type being analyzed, remain uninfected through the vaccination series, and receive the appropriate 3 doses of the vaccine without protocol violations. Such individuals approximate HPV-uninfected pre-adolescents for that particular vaccine-HPV type, but are actually at risk of acquiring infection and disease and therefore represent a suitable population to evaluate the efficacy of the HPV vaccine for that particular HPV type. Efficacy calculated in this population (termed the per-protocol efficacy [PPE] or according-to-protocol [ATP] population in vaccine efficacy trials) is interpretable as prophylactic HPV vaccine efficacy. It has been consistently shown in clinical trials that prophylactic HPV vaccine efficacy approached 100% for HPV types covered by a particular HPV vaccine [5], [8], [9], [16].

Conversely, efficacy against a particular vaccine-HPV type that is calculated in a population of individuals who are HPV-infected for that particular type during the vaccination period is interpretable as therapeutic efficacy (i.e., a measure of whether the vaccine can clear existing infection). In HPV VLP vaccine clinical trials, no therapeutic efficacy has been demonstrated [5], [7], [8], [9], [10].

To illustrate this point, analysis populations previously used in efficacy trials of the 4vHPV vaccine are shown in Table 1; these include the PPE and ITT populations as well as the ITP population, a modified ITT population that includes only subjects not HPV-infected prior to vaccination [8], [9], [17]. An example of efficacy analysis of 4vHPV vaccine to prevent the endpoint of CIN2+ associated with HPV type 16 or 18 based on these 3 analysis populations in the Female United to Unilaterally Reduce Endo/Ectocervical Disease (FUTURE) II study is shown in Table 2 [8]. The estimated vaccine efficacy in the prophylactic analysis populations (PPE, ITP) is substantially higher than in the ITT population because the incidence of the efficacy endpoint in the 4vHPV vaccine group is much higher in the ITT population than in the PPE and ITP population. As noted, subjects who were infected prior to vaccination are included in the ITT population and excluded from the ITP population. Thus, most reports of disease in the ITT population came from subjects who were infected prior to vaccination.

Table 1

Efficacy analysis populations used in the 4vHPV vaccine clinical program.

Efficacy Analysis Populations
Per-protocol efficacy (PPE)	The primary efficacy analysis population. All subjects in the PPE population were required to be seronegative to the relevant HPV type at Day 1 and PCR-negative to the relevant HPV type from Day 1 through Month 7, have received all 3 vaccinations within 1 year, and have no protocol violation.
Intention-to-prevent (ITP)a	A secondary, broader analysis population including those who received at least 1 vaccination and had efficacy follow-up. Moreover, like for the PPE population, subjects in the ITP population were required to be seronegative and PCR-negative to the relevant HPV type at Day 1. The ITP analysis differed from the PPE analysis in that it included protocol violators and subjects who became infected with a vaccine HPV type during the vaccination period.
Intention-to-treat (ITT)	The ITT analysis include all those who received at least 1 vaccination an had efficacy follow-up. It includes protocol violators and those who were infected with HPV at baseline

In prior publications, the ITP population was given various names including ‘unrestricted susceptible population’, ‘naïve to the relevant type population’, or ‘HPV naïve type-specific population’ [8], [9], [17], [20].

Table 2

4vHPV vaccine efficacy against CIN 2 + lesions associated with HPV 16 or HPV 18 [8].

Population	4vHPV Vaccine (N = 6087)			Placebo (N = 6080)			Efficacy % (95% CI)
	Total subjects	No. of cases	Ratea	Total subjects	No. of cases	Ratea
PPE	5305	1	<0.1	5260	42	0.3	98 (86–100)
ITP	5865	3	<0.1	5863	62	0.4	95 (85–99)
ITT	6087	83	0.5	6080	148	0.8	44 (26–58)

The rate is the number of subjects with the endpoint per 100 person-years at risk.

Impact of prevalence of baseline HPV infection on the estimate of vaccine efficacy in the ITT population

Since HPV VLP vaccines are prophylactic but not therapeutic vaccines, HPV-related disease prevention is expected among those not HPV-infected but not expected among those HPV-infected during the vaccination period. In statistical analysis parlance, existing HPV infection status (infected versus not-infected) at the time of vaccination is a clearly established subject characteristic that has an interaction with vaccine efficacy. Given that the ITT analysis population includes both HPV-infected and not HPV-infected at the time of vaccination, the estimate of HPV vaccine efficacy in an ITT analysis population, which is commonly interpreted as a measure of ‘overall’ vaccine efficacy, is in reality a mixture of prophylactic and ‘therapeutic’ efficacy. In fact, there is no therapeutic efficacy and none is claimed, so any prevalent infection or disease simply dilutes the true prophylactic efficacy and does not contribute to the understanding of the effectiveness of the vaccine. Efficacy in the ITT population simply reflects the amount of prevalent infection in a particular population of study subjects. Another interesting observation is that when vaccine is used in 11-year-old children, the ITP and ITT analysis are equivalent because of the absence of prevalent infection or disease in this population. This is why the ITP analysis is vital to the efficacy metric because in sexually active individuals, where efficacy can be measured, the ITP population best approximates the situation expected in uninfected young adolescents. In contrast, the measure of overall vaccine efficacy calculated in the ITT analysis population via a ‘pooled’ analysis (i.e., without regard to adjustment for HPV infection status when there is a clear interaction between HPV infection status and vaccine efficacy) has no meaningful and practical interpretation and is not an appropriate statistical analysis approach.

On the other hand, one might argue that the appropriate solution to the problem of prevalent infection is to recruit study subjects who are not infected with the HPV types under study. Such pre-screening is impractical for several reasons. Studying the safety of the vaccine administered to subjects who are prevalently infected is an important question that requires study in the clinical trials, as well as to demonstrate efficacy against HPV types to which study participants were not infected. Additionally, developing an HPV vaccine that requires pre-screening for HPV infection would render any vaccination program in a general population infeasible. The clinical development program of such a vaccine is designed to demonstrate prophylactic efficacy and safety on an HPV type-specific basis and support the development of a vaccine suitable for a real-world vaccination program. It should not be designed to specifically recruit a study population for a clinical trial because it creates a favorable ITT analysis.

Additionally, in HPV vaccine efficacy clinical trials where a primary efficacy endpoint such as incidence of high-grade cervical disease (CIN2+) takes several years to develop and be observed, the characteristic of being HPV-infected at baseline is magnified over the duration of a clinical trial because it is just such a characteristic that contributes to the development of CIN2+ over the course of a clinical trial, and ultimately contributes to accumulation of the primary efficacy endpoint. This type of impact in HPV vaccine efficacy trial, where subgroups in an ITT population who have no expected benefit from therapy yet actually contribute to increasing the count of primary efficacy endpoint, is opposite of the usual impact of such ITT subgroups in an ITT analysis population of drug studies. In a typical drug study where the endpoint is some measure of “cure”, subgroups in an ITT population who have no expected benefit from therapy do not contribute to increasing the count of subjects who were cured. Thus, in such drug studies, inclusion of subgroups in an ITT population who have no expected benefit from therapy will not profoundly affect a measure of treatment effect such as percent risk reduction. By contrast, in an HPV vaccine efficacy study, including subgroups of HPV-infected subjects in an ITT population who have no expected benefit from vaccination will profoundly affect the measure of vaccine efficacy. For example, the rate of CIN2+ in an ITT population that is observed over an approximately four-year clinical trial is partly a result of the background rate of infection and disease that is characteristic of the ITT population at the start of the clinical trial. In a trial that enrolls a high proportion of HPV-infected subjects in both the vaccine and placebo groups, such HPV-infected subjects will cause an accumulation of the CIN2+ endpoint in both the vaccine and placebo groups. Thus, the ITT analysis estimate of the vaccine efficacy will no longer reflect the vaccine's prophylactic efficacy because including endpoints from HPV-infected subjects does not preserve the unbiased comparability of vaccine and placebo groups with respect to the evaluation of prophylactic efficacy.

Generally, in an ITT analysis where vaccine efficacy is estimated without adjustment for the proportions of the study population who were and were not HPV-infected at enrollment, the ITT estimate of ‘overall’ vaccine efficacy would be closer in value to the expected efficacy in the subgroup that contributed the majority of the endpoint cases. If the baseline HPV-uninfected subgroup (i.e., ITP population) contributed the majority of endpoint cases, the ‘overall’ vaccine efficacy estimate would be closer in value to the prophylactic vaccine efficacy estimate. If the HPV-infected subgroup contributed the majority of endpoint cases, the ‘overall’ vaccine efficacy estimate would be closer in value to the non-existent therapeutic vaccine efficacy estimate. Note that in the pooled analysis of vaccine efficacy, the sample size of the subgroup does not matter; the relative contribution of the subgroup in terms of event counts determines what the ‘overall’ vaccine efficacy estimate in the ITT analysis would reflect. The profound impact of baseline prevalence of infection and post baseline incidence of disease on the estimate of vaccine efficacy in a particular clinical trial study population being analyzed based on the ITT analysis approach was also illustrated by others (Table 3) [18].

Table 3

Efficacy estimates for any CIN3 or AIS, regardless of causal HPV type, for two populations in the Gardasil clinical trials [21].

	Population with lower HPV prevalence and lower incidence of disease (HPV-naive)a		Population with higher HPV prevalence and higher incidence of disease (intention-to-treat)b
	Vaccine (n = 4616)	Placebo (n = 4680)	Vaccine (n = 8562)	Placebo (n = 8598)
Positive to ≥l HPV type at day 1 or abnormal cytology at day 1	No	No	Yes, ∼47% with infection or disease at day 1	Yes, ∼47% with infection or disease at day 1
Incidence of any CIN3 or AIS (cases per 100 person-years at risk)	0.22	0.39	0.81	0.98
Efficacy estimate (%) for any CIN3 or AIS (95% CI)c	43(13–63)	–	18 (2–31)	–
Estimated number of disease cases prevented annually per 100,000 vaccinated women (95% CI)d	170 (50–280)	–	180 (30–330)	–

AIS: Adenocarcinoma in situ; CIN3: Cervical intraepithelial neoplasia grade 3; HPV: Human papillomavirus.

Copyright permission has been granted for reuse of Table 3.

This population was restricted to subjects who received at least one vaccination and. at enrollment: were seronegative and DNA negative to HPV6, 11, 16 and 18; were DNA negative to the ten nonvaccine types, including HPV31, 33, 35, 39, 45, 51, 52, 56, 58 and 59; and had a normal Pap test result.

Intention-to-treat population was all subjects who received at least one injection of quadrivalent HPV vaccine or placebo and had follow-up, regardless of the presence of HPV infection or HPV-related disease at enrollment.

Efficacy estimates for the HPV-naive and intention-to-treat populations were very different at 43 and 18%, respectively.

The number of disease cases prevented in the HPV-naive and intention-to-treat populations was similar at 170 and 180, respectively.

In summary, the ITT analysis estimate of vaccine efficacy is neither a meaningful metric for assessing the vaccine's prophylactic efficacy, nor a good representation of the impact in reducing the absolute risk of disease related to HPV types that the vaccine is designed to prevent. Furthermore, comparison of ITT analysis estimates of vaccine efficacy across clinical trial programs of different vaccines is highly misleading because it reflects the population recruited to the study, rather than the effect of vaccine efficacy.

Discussion

Proposed metric and analysis approach to estimate the benefit of vaccination in the general population

Approach #1: assess vaccine efficacy in ITP population instead of ITT population

If vaccine efficacy in an ITT population is not an accurate metric for assessing the vaccine's prophylactic impact in the general population, then what metric and analysis population or analysis approach should be used? In contrast to an ITT analysis, the ITP analysis is a more appropriate approach for prophylactic vaccines, particularly for multivalent HPV vaccines. An ITP analysis approach in the context of HPV vaccine trials would follow the ITT analysis approach insofar as everyone assigned to receive vaccination is included in the analysis and categorized in groups based on the initially assigned vaccination group regardless of actual vaccination regimen received. However, an ITP analysis population for a specific HPV type would only include those subjects randomized to the experimental or control group who received at least one vaccination dose and are uninfected for the specific HPV type, i.e., DNA negative for this HPV type, at the time of initial vaccination.

The impact of including subjects in the analysis population who have no potential to derive benefit from therapy is profoundly different in therapeutic drug studies compared to prophylactic vaccine studies. In drug studies where the endpoint is typically related to transitioning from a diseased-state to a disease-free-state, including subjects who have no potential to derive benefit from therapy (e.g., subjects randomized but did not receive drug) does not affect the count of subjects who transitioned from a diseased-state to a disease-free-state and will not inflate the incidence of the study primary endpoint. Consequently, a drug that is 100% efficacious relative to a control group has a chance of being detected as such in an ITT analysis that includes subjects who did not receive a drug. In prophylactic HPV vaccine studies where the endpoint is typically related to transitioning from a disease-free-state to a diseased-state, including subjects who have no potential to derive benefit from therapy (e.g., subjects who did not receive vaccination, or subjects who are HPV-infected at the time of vaccination) have non-negligible impact in the count of subjects who transition to the diseased-state and will inflate the incidence of the study primary endpoint. Consequently, a prophylactic vaccine that is truly 100% efficacious relative to a control group may not be detected as such in an ITT analysis that includes subjects who are HPV-infected at the time of vaccination.

The primary goal of an ITT analysis approach is to attempt to preserve the unbiased comparability of the groups being compared that is afforded by randomization. Drug studies have sample sizes that are relatively small compared to vaccine efficacy studies. Perturbations in the unbiased comparability of groups being compared resulting from exclusions of subjects from the ITT analysis population are more likely in drug studies compared to vaccine efficacy studies. HPV vaccine efficacy studies have large sample sizes, typically several thousands of subjects. On the contrary, perturbations in the unbiased comparability of groups being compared resulting from exclusions of subjects from the ITT analysis population are less likely with large sample sizes.

An ITP analysis population for an HPV vaccine efficacy study will not necessarily negatively impact the unbiased comparability of the vaccination groups being compared that was afforded by randomization. On the contrary, an ITP analysis population eliminates the one subject characteristic that has been clearly demonstrated to have an interaction with vaccination effect, i.e., HPV-infection at the time of vaccination. Thus, an ITP analysis population eliminates one factor that introduces ambiguity in the interpretation of the vaccine efficacy metric that is calculated from an ITT analysis population of HPV vaccine efficacy studies.

Approach #2: assess absolute risk reduction instead of vaccine efficacy in ITT analyses

As previously mentioned, vaccine efficacy is the proportional amount of reduction in risk of disease in the control group studied in a clinical trial. Efficacy is expressed as a percent of the risk of disease that can be prevented in the particular control group studied in a clinical trial. When using an ITT analysis, there are even better methodologically approaches than vaccine efficacy for assessing the impact of prophylactic HPV vaccines in reducing the risk of disease or number of cases prevented in the general population. Absolute risk reduction (ARR) for example, which is calculated as risk of disease in a vaccinated population minus the risk of disease in an unvaccinated population, is a more relevant metric for assessing the impact of prophylactic vaccines in reducing the absolute risk of disease in the general population. Also, ARR is not profoundly affected by background rates of infection or disease in the different populations in a way that vaccine efficacy is. An example of the comparison of efficacy and ARR is provided in Table 3 from Haupt and Sattler [18]. Using actual data from the Gardasil clinical trials, efficacy against CIN grade 3 or adenocarcinoma in situ irrespective of HPV is 43% in a population of mostly HPV-uninfected women (low prevalence and low incidence of HPV) resulting in approximately 170 cases prevented annually per 100,000 vaccinated women. Vaccinating a population with high prevalence and high incidence, the classical ITT analysis, results in observed efficacy of 18%, while the estimated number of cases prevented per 100,000 vaccinated women is 180, which is essentially the same as cases prevented where efficacy was estimated to be 43%.

Conclusion

An ITP, not ITT, analysis approach in a HPV vaccine efficacy trial is relevant for estimating prophylactic HPV vaccine efficacy. Clinical trial study populations do not always represent general populations of women and differ by individual studies due to inclusion/exclusion criteria, specific age cluster, background rates of prevalent infection or disease and other criteria that serve to demonstrate the vaccines' efficacy in an uninfected group of women. In HPV vaccine studies, ITT analyses that purport to provide measures of overall vaccine efficacy in the general population are not helpful and potentially misleading, especially given the prophylactic nature of the vaccine. Efficacy in the ITT population simply reflects the amount of prevalent infection in a particular population of study subjects.

The dramatic beneficial effect that has been seen in real life situations, such as in Australia, where high vaccine uptake in young sexually active women has resulted in marked reductions in HPV prevalence, genital wart incidence and CIN lesions supports the observations that important benefit of HPV vaccination can accrue in such a sexually active population [19]. Furthermore, the common analysis approach used in HPV vaccine clinical studies of censoring a subject at the time of CIN2+ due to prevalent infection during estimation of vaccine efficacy markedly limits downstream ascertainment of new lesions prevented due to HPV types in the vaccine. Such censoring in the clinical trials does not methodologically affect or hinder continued effectiveness in a general population of women. In other words, those women are protected and contribute to reduction in cases if they are not infected at the time of vaccination to the relevant HPV type and there is no censoring of data. Therefore, real-world population data may represent the best approximation of an overall L1 VLP HPV vaccine effectiveness because it captures the absolute and actual reduction in infection and disease throughout a given population.

Contributions

Alfred Saah, Oliver Bautista, Alain Luxembourg and Gonzalo Perez contributed to the research and writing of this article.

Conflicts of interest

Alfred Saah, Oliver Bautista, Alain Luxembourg and Gonzalo Perez are employees of Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA and may own stock and/or stock options in the company.

Funding

This research was funded by Merck & Co., Inc., Kenilworth, NJ, USA (sponsor). Although the sponsor formally reviewed a penultimate draft, the opinions expressed are those of the authorship and may not necessarily reflect those of the sponsor. All co-authors approved the final version of the manuscript.