Adjuvant-containing control arms in pivotal quadrivalent human papillomavirus vaccine trials: restoration of previously unpublished methodology.

PURPOSE: Trustworthy reporting of quadrivalent human papillomavirus (HPV) vaccine trials is the foundation for assessing the vaccine's risks and benefits. However, several pivotal trial publications incompletely reported important methodological details and inaccurately described the formulation that the control arms received. Under the Restoring Invisible and Abandoned Trials initiative (RIAT), we aim to restore the public record regarding the content and rationale of the controls used in the trials.
METHODS: We assembled a cohort (five randomised controlled trials) described as placebo-controlled using clinical study reports (CSRs) obtained from the European Medicines Agency. We extracted the content and rationale for the choice of control used in each trial across six data sources: trial publications, register records, CSR synopses, CSR main bodies, protocols and informed consent forms.
RESULTS: Across data sources, the control was inconsistently reported as 'placebo'-containing aluminium adjuvant (sometimes with dose information). Amorphous aluminium hydroxyphosphate sulfate (AAHS) was not mentioned in any trial registry entry, but was mentioned in all publications and CSRs. In three of five trials, consent forms described the control as an 'inactive' substance. No rationale for the selection of the control was reported in any trial publication, register, consent form, CSR synopsis or protocol. Three trials reported the rationale for choice of control in CSRs: to preserve blinding and assess the safety of HPV virus-like particles as the 'safety profile of (AAHS) is well characterised'.
CONCLUSIONS: The stated rationale of using AAHS control-to characterise the safety of the HPV virus-like particles-lacks clinical relevance. A non-placebo control may have obscured an accurate assessment of safety and the participant consent process of some trials raises ethical concerns. TRIAL REGISTRATION NUMBERS: NCT00092482, NCT00092521, NCT00092534, NCT00090220, NCT00090285. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

INTRODUCTION

Randomised, placebo-controlled clinical trials are considered the gold standard for evaluating new vaccines. To evaluate the vaccine’s efficacy and safety, the manufacturer of quadrivalent human papillomavirus (qHPV) vaccine conducted multiple clinical trials including around 30 000 females and males. The vaccine was approved in 2006. Today, citing the vaccine’s efficacy and extensive safety profile, increasing HPV vaccine coverage is a high priority for many national governments and the WHO.1–3

Multiple trials of qHPV vaccine are reported as ‘placebo-controlled.’4–8 However, participants in the ‘placebo’ arms of these pivotal trials received an injection-containing amorphous aluminium hydroxyphosphate sulfate (AAHS), a proprietary adjuvant. AAHS is used in qHPV vaccine to boost immune response, but the rationale for adding AAHS to the ‘placebo’ is not reported in publications of these trials4–8 and is contrary to the advice of the public health bodies and regulators. The WHO, for example, recommends control recipients in trials testing an unlicensed, experimental vaccine receive either an inert substance or an approved efficacious vaccine.9 Similarly, the European Medicines Agency (EMA), which provides specific guidance on the clinical evaluation of vaccines, recommends the use of a licensed vaccine without an effect on the target disease if a placebo cannot be used.10

In addition, the efficacy and safety analyses of these five qHPV vaccine trials were conducted as if the trials were controlled with inert placebo when they were not. None of the key publications for these trials, which have been used to inform regulatory and health decision making, appear to discuss how AAHS-containing control could affect the interpretation of results.4–8

We consider the omission in journal articles, of any rationale for the selection of AAHS-containing control, to be a form of incomplete reporting of important methodological details, and believe the rationale must be reported. We also consider that the use of the term ‘placebo’ to describe an active comparator like AAHS inaccurately describes the formulation that the control arm participants received, and constitutes an important error that requires correction. If trial participants were told they could receive ‘placebo’ (widely defined as referring to an ‘inactive’11 12 or ‘inert’9 substance) without being informed of all non-inert contents of the control arm injection, this raises ethical questions about trial conduct as well.

In January and February 2019, we publicly declared our intention to systematically correct the record for these trials13 14 in accordance with the principles of the Restoring Invisible and Abandoned Trials (RIAT) initiative, of which some of us are founders.15 RIAT was founded with the intent of restoring the text of published biomedical articles where there is clear evidence of distortion or omission (as in the case of reporting bias or non-publication of clinical trials) and the original trial authors and/or sponsors fail to take any corrective action (thereby abandoning their responsibilities). While RIAT projects to date have focused on restoring trials through a reanalysis of all study data for a given trial,16–19 this project differs in its scope in two ways: first, we are focused on multiple trials in a single manuscript; second, this restoration is restricted to a specific aspect of the trials’ methodology (the choice of control). We believe this ‘focused’ RIAT is important given the considerable implications that the choice of control has on the interpretation of trial results, and the fact that the problem is not confined to a single trial but rather spans a trial programme. Other criticisms of the reporting in qHPV trial publications (e.g., selective outcome reporting) have been made,20 but we do not address those concerns here. Consistent with the RIAT approach, we wrote to the corresponding authors of all five publications as well as the sponsor requesting that they correct their publications. None contested our concerns nor indicated any intention to correct the record. As such, we consider these trials abandoned and hereby restore the record using data obtained under freedom of information requests to the EMA according to our prespecified protocol.21

Subjects and methods

Research design

Review of text describing the content and rationale for choice of control across published and unpublished data sources.

Data sources, searches and restoration sample

Since 2010, transparency policies at the EMA have allowed independent investigators to obtain access to clinical study reports (CSRs) and other clinical data once protected as commercially confidential.22 CSRs are reports structured according to international guidelines,23 written by sponsors and submitted to regulators, generally as part of marketing authorisation applications, and provide far more detail than journal publications.24 In 2015, the EMA expanded its efforts to proactively publish CSRs to its website following regulatory decisions,25–27 a practice Health Canada also adopted as of 2019.28

The cohort of trials potentially eligible for restoration was all clinical trials of qHPV vaccine and nonavalent HPV vaccine for which we obtained CSRs from the EMA. These were received in response to a May 2014 freedom of information request. The process of obtaining CSRs has been previously described in an Index study29 and Analysis article.30 All trials evaluating qHPV vaccine and nonavalent HPV vaccine for which CSRs were obtained by 1 November 2018 were eligible for inclusion in this restoration.

Each CSR was reviewed by two independent assessors (with conflicts resolved by discussion or third party) for possible inclusion against the following criterion: trial randomised control group participants to aluminium-containing adjuvant (as a control). We excluded trials which used an approved vaccine as the control.

Identification of matching trial publications

We aimed to identify the most impactful, company-authored trial publication for all trials in our sample using the following approach:

We identified the NCT number for each trial by searching for the trial ID, obtained from the CSR, on ClinicalTrials.gov. We then compared trial characteristics (study population, intervention, comparator, name of primary outcome measure(s) and number of participants) with the CSR and a published index of HPV vaccine trial programme29 to confirm that the correct NCT record was identified.

From the ClinicalTrials.gov registry entry for each trial, we recorded all publications (in all categories) listed under the ‘Study Details’ tab.

Using the DOI of these publications, we identified the most cited publication of each trial using the Scopus database and obtained the article’s full-text PDF and supplementary material.

This process was carried out by two independent reviewers with discrepancies resolved by discussion or third-party consultation.

Determination of under-reporting and misreporting

For each trial using a concurrent aluminium-containing adjuvant as a control, we reviewed the publication to determine the potential presence of under-reporting of the rationale for using such a control and misreporting of the control intervention (i.e., describing the control as ‘placebo’).

Data extraction

For each trial using six data sources (CSR synopsis, CSR main body, trial protocol, informed consent form, trial publication and trial register entry), we recorded (1) the phrases used to describe the comparator to qHPV vaccine; (2) the rationale for using aluminum-containing adjuvant as a control, if present and (3) all listed contents (ingredients) of formulation received by intervention and control arms.

To enable an exhaustive list of all descriptions of the comparator across all document types, especially CSRs which were often thousands of pages long, we searched for the terms placebo, control, comparator, alum, AAHS, adjuvant and MAA (Merck Aluminum Adjuvant). We categorised the verbatim phrases into three broad categories: phrases that described a placebo (with no mention of adjuvant), phrases that described a placebo containing adjuvant and phrases describing a placebo containing adjuvant as well as adjuvant dose information. We recorded the frequency of terms used to describe the formulation of the control arm and its ingredients, by data source.

Extractions were independently carried out by two assessors. There were no discrepancies, and a consolidated analysis data set was created.

Results

We obtained 14 trials from the EMA (eight qHPV vaccine, six nonavalent HPV vaccine), of which five met our prespecified inclusion criteria (V501-012, V501-013, V501-015, V501-019 and V501-020; figure 1). All included trials compared qHPV vaccine to aluminium-containing adjuvant. Corresponding register and publication information for each trial are listed in table 1.

Figure 1

Summary of Trial Search. CSR, clinical study reports; EMA, European Medicines Agency; HPV, human papillomavirus.

Table 1

Characteristics of trials being restored

Trial ID/name	ClinicalTrials.gov registration #	Most cited trial publication	Participants enrolled*	Number of citations†
V501-012 ‘FUTURE 1’	NCT00092482	Garland et al 7	3882 women, aged 16–24	32
V501-013 ‘FUTURE 1’	NCT00092521	Garland et al 8	5455 women, aged 16–24	1312
V501-015 ‘FUTURE 2’	NCT00092534	FUTURE II Study Group4	12167 women, aged 15–26	1440
V501-019 ‘FUTURE 3’	NCT00090220	Muñoz et al 5	3819 women, aged 21–46	318
V501-020	NCT00090285	Giuliano et al 6	4065 men, aged 15–27	593

*Details taken from the clinical study report.

†Citation count from SCOPUS database as on 14 December 2018.

Describing the control arm

There was heterogeneity in the verbatim phrases used to describe the control arm across all trials and types of documents (table 2).

Table 2

Phrases used to describe the control arm (proportion of trials)

Phrases used	ICF	CSR			Register	Publication
		Synopsis	Main body	Protocol		Abstract	Full text
Placebo	5/5	5/5	5/5	5/5	4/5	4/5	5/5
‘Placebo’ or ‘matching placebo’	5/5	5/5	5/5	5/5	4/5	4/5	5/5
‘Placebo (which is a dose that contains no active ingredients)’	4/5	0/5	0/5	0/5	0/5	0/5	0/5
‘Placebo … an inactive substance’ or ‘placebo … an inactive solution’	3/5	0/5	0/5	0/5	0/5	0/5	0/5
‘Placebo HPV Vaccine’	3/5	0/5	0/5	0/5	0/5	0/5	0/5
Placebo with adjuvant	0/5	1/5	5/5	4/5	0/5	0/5	5/5
‘Placebo with adjuvant’	0/5	0/5	2/5	0/5	0/5	0/5	0/5
‘Aluminum adjuvant placebo’ or ‘Alum-placebo’ or ‘aluminum-placebo’ or ‘aluminum-containing placebo’ or ‘visually indistinguishable aluminum-(or ‘AAHS’)containing placebo’ or ‘matched aluminum-placebo’ or ‘placebo (aluminum adjuvant)’	0/5	1/5	4/5	4/5	0/5	0/5	5/5
Placebo with adjuvant and dose	3/5	3/5	3/5	5/5	0/5	0/5	0/5
‘Placebo’-containing 225 ug of aluminium adjuvant	0/5	2/5	1/5	4/5	0/5	0/5	0/5
‘Placebo’ or ‘matched placebo’-containing Merck standard aluminium diluent (225 mcg alum), with or without mention of normal saline	0/5	1/5	0/5	2/5	0/5	0/5	0/5
‘Placebo’-containing 225 mcg of aluminium as AAHS, with or without mention of normal saline	0/5	1/5	2/5	0/5	0/5	0/5	0/5
‘Inactive solution (containing aluminium 225 mcg/dose)’	3/5	0/5	0/5	0/5	0/5	0/5	0/5

Normal saline = USP (NaCl 0.9%).

AAHS, amorphous aluminum hydroxyphosphate sulfate; CSR, clinical study report; ICF, informed consent form.

The publication abstracts and register entries for four trials only referred to the control arm as ‘placebo’ or ‘matching placebo’ (Garland et al 8/NCT00092521, FUTURE II Study Group4/NCT00092534, Muñoz et al 5/NCT00090220, Giuliano et al 6/NCT00090285). The publication abstract and register entry for one trial (Garland et al 7/NCT00092482) did not mention the existence of a control arm.

The informed consent forms for two trials (V501-019 and V501-020) did not provide participants with any information regarding the contents of the control arm. The informed consent forms for three trials (V501-012, V501-013 and V501-013) informed participants that they may be exposed to an ‘inactive solution (containing aluminium 225 mcg/dose)’.

The CSRs of all trials described the control arm as a ‘placebo’-containing 225 mcg of aluminium. While all CSRs stated the type of aluminium adjuvant (AAHS), the information was not reported in a consistent location (i.e., CSR synopsis, main body and protocol). A summary of reported ingredients, by data source, is presented in table 3.

Table 3

Ingredients of control arm, by trial document type

Trial	Clinical study report	Informed consent form	Register	Publication
V501-012	225 mcg of aluminium as AAHS in normal saline	Inactive solution (containing aluminium 225 mcg/dose)	Control intervention not listed	Aluminium-containing placebo
V501-013	225 mcg of aluminium as AAHS in normal saline	Inactive solution (containing aluminium 225 mcg/dose)	No ingredients listed; only states ‘placebo’	Aluminium-containing placebo
V501-015	225 mcg of aluminium as AAHS in normal saline	Inactive solution (containing aluminium 225 mcg/dose)	No ingredients listed; only states ‘placebo’	Aluminium-containing placebo
V501-019	Merck standard aluminium diluent (225 mcg alum) in normal saline	No ingredients listed. Only states: ‘The HPV vaccine placebo contains no active vaccine’	No ingredients listed; only states ‘placebo’	Aluminium-containing placebo
V501-020	225 mcg of aluminium as AAHS in normal saline	No ingredients listed. Only states: ‘placebo, an injection that looks the same as the vaccine but has no active ingredient’	No ingredients listed; only states ‘placebo’	AAHS-containing placebo

Normal saline = USP (NaCl 0.9%).

AAHS, amorphous aluminum hydroxyphosphate sulfate.

We assessed the frequency of terms used to describe the control for the trial with the most highly cited publication in our study (V501-015). In the publication, the control was referred to as a placebo (with no mention of adjuvant) 67 times and one time as a placebo with adjuvant.4 In the CSR, the control was referred to as a placebo 1450 times versus 326 times as placebo with adjuvant, and across six informed consent forms (US and international versions of the main consent form, child assent form and pregnancy consent addendum), 54 versus 2, respectively. Finally, in the register entry, the control was referred to as a placebo 50 times versus 0 times as a placebo with adjuvant.

Rationale for aluminium-containing control

The rationale for the selection of an AAHS-containing control was reported in the CSR of three trials (V501-012, V501-013 and V501-015). No rationale was provided in any document for two trials (V501-019 and V501-020). In all five trials, the protocol, publication, register entry, informed consent form and CSR synopsis did not contain a rationale for the selection of the control (table 4).

Table 4

Rationale for aluminium-containing control (verbatim quotes)

Trial	Publications, registers, study protocols, ICFs, and CSR synopses	Clinical study reports (CSR main body)
V501-012	None provided	‘Aluminum adjuvant was chosen as the appropriate control for the qHPV vaccine for the following reasons: The inclusion of aluminum adjuvant in both vaccine and placebo preserved the blinding of the study because it allowed the vaccine and placebo to be visually indistinguishable; and The safety profile of Merck’s aluminum adjuvant is well characterised. On the other hand, the safety profile of the HPV 6, 11, 16 and 18 L1 VLPs required further evaluation in humans. By using placebo that contained a dose of aluminum adjuvant that was identical to the dose included in the qHPV vaccine, it was possible to assess the safety profile attributable to the HPV 6, 11, 16 and 18 L1 VLP component of the vaccine.’
V501-013	None provided
V501-015	None provided
V501-019	None provided	None provided
V501-020	None provided	None provided

Normal saline = USP (NaCl 0.9%).

CSR, clinical study report; ICF, informed consent form; qHPV, quadrivalent human papillomavirus; VLP, virus-like particle.

The three studies that reported a rationale for the selection of an AAHS-containing control provided identical reasons: (1) to preserve blinding and (2) to assess the safety of HPV virus-like particles given that the ‘safety profile of (AAHS) is well characterised’.

Discussion

On its website, qHPV vaccine’s sponsor defines a placebo as ‘an inactive pill, liquid, or powder that has no treatment value’.11 This definition is consistent with the decades old notion of placebos as pharmacologically inert substances used to obtain unbiased assessments in experimental research.31

The five pivotal trials of qHPV vaccine that were the subject of this restoration used a control with aluminum-containing adjuvant. This was stated in the journal publications of all trials. But contrary to what the original investigators reported, the trials were not placebo-controlled. The inclusion of AAHS in the control arm means that the control should not be reported as a placebo since AAHS is not inactive. While the original FDA-approved prescribing information in 2006 reported the AAHS-containing control as ‘placebo’, this was updated in September 2008, and all such references were removed and replaced with ‘AAHS control’.32–34

Inaccurate use of the term placebo is not confined to qHPV vaccine trials. As no regulations govern placebo composition, researchers have documented a diversity in what gets labelled a placebo in research, and that depending on the experimental arm, some so-called placebos may have effects that influence study outcomes.35

With respect to adjuvants in vaccines, the FDA has noted that ‘adjuvants have their own pharmacologic activity, which may affect both the immunogenicity and the safety of vaccines. Adverse reactions may include local reactions such as pain, swelling, injection site necrosis, and granulomas. Systemic reactions may include nausea, fever, arthritis, as well as potential immunotoxic reactions. Unexpected, rare events may also occur’.36

The FDA has also stated that ‘the evaluation of safety of an adjuvanted vaccine needs to include special safety considerations’, which may include data ‘derived by comparing the adjuvanted vaccine to a placebo or the unadjuvanted vaccine antigen, if feasible’.36 The extent to which the evaluation of qHPV vaccine included such special safety considerations is debatable given the presence of adjuvant in the control.

The implications of AAHS controlled trials on an accurate assessment of the safety and efficacy of qHPV are important. It is possible that the presence of AAHS in both arms of the trials masked AAHS’s reactogenic potential thereby obscuring any differential rates in harms, leading to an underestimate in harms. The qHPV vaccine trial results are consistent with these statements. Across the five trials using AAHS in the control, the absolute risk of injection-site adverse events was only 6%–13% higher in qHPV vaccine patients. By contrast, in study V501-018, which used a control lacking AAHS, the absolute risk increase was 25% (table 5). It is also possible that AAHS’s stimulating properties (which to our knowledge have never been studied on their own) provoked an immune response which may have been protective against endpoints studied in the trial, thereby leading to an underestimation of the effectiveness of qHPV vaccine. Because all the five trials lacked a true inert comparator arm, the trial data cannot resolve these questions.

Table 5

Injection-site adverse experiences between Gardasil and control arms

Trial	Gardasil	Control	Difference	Source
V501-012	87.9%	78.6% (with AAHS)	9.3%	CSR p.33
V501-013	88.0%	79.8% (with AAHS)	8.2%	CSR p.13
V501-015*	84.4%	77.9% (with AAHS)	6.5%	Publication table 4
V501-019	76.7%	64.2% (with AAHS)	12.5%	CSR p.566
V501-020	60.1%	53.7% (with AAHS)	6.4%	CSR p.348
V501-018	75.3%	50.0% (without AAHS)	25.3%	CSR p.33

*The CSR for V501-015 reports injection-site adverse experiences as 8.8% (qHPV) versus 7.6% (control), but these figures are likely an underestimate as injection-site adverse experiences were only collected for participants in the Non-Serious Adverse Experience Substudy, and data for this subpopulation were reported in the publication but not CSR.

AAHS, amorphous aluminium hydroxyphosphate sulfate; CSR, clinical study reports.

It is not clear whether other trials can establish qHPV vaccine’s true safety and efficacy profile. The FDA instructed the sponsor to conduct a safety study with a control not containing aluminium, according to the company’s submission to Japanese regulators,37 and the CSR for trial V501-018 states that it was the only qHPV vaccine trial to do this. In contrast to other studies, the FDA medical officer who reviewed the sponsor’s marketing application referred to this trial’s control as a ‘true placebo’.38 However, V501-018’s short follow-up of control arm participants (30 months), small sample (597 receiving control) and younger age relative to other trials makes it inadequate to resolve safety questions.

Our study raises serious questions about the ethics of consent in the trials. The informed consent forms for the trials described the placebo as an ‘inactive solution’ and ‘a dose that contains no active ingredients’. This likely misled many trial participants who would have reasonably expected the control did not contain a pharmacologically active adjuvant.

Our restoration makes public, for the first time, the sponsor’s rationale for using AAHS as a control. The manufacturer gave two reasons: first, to ensure that the control was not visually distinguishable, and second, to assess the safety profile of the HPV virus-like particles as the ‘safety profile of Merck’s aluminium adjuvant is well characterised’.

The fact that the qHPV vaccine and nonavalent HPV vaccine clinical trial programme included two randomised, double-blind trials that did not use an AAHS-containing control demonstrates that there was no necessity to use AAHS to maintain blinding. In these trials (V501-018 and V503-006), unblinded study personnel who were otherwise not involved in the study prepared and administered injections.

Second, the safety profile of AAHS is not well characterised. We are unaware of any randomised trial directly evaluating AAHS against placebo; a network meta-analysis is underway that aims to use extant randomised controlled trial (RCT) data to determine the safety profile of the AAHS component of qHPV vaccine.39 Moreover, while other vaccines contain AAHS (e.g., Recombivax and VAQTA), such vaccines cannot be used to characterise AAHS safety as ‘it cannot be assumed that an adjuvant that is safe in one vaccine with a given antigen will be safe when added to another vaccine’.40 The unknown consequences of aluminum-containing adjuvants, more generally, is concerning. A Cochrane review is currently underway to determine whether any RCTs exist which evaluated the effects of aluminium adjuvants compared with placebo or no intervention.41

Finally, the stated rationale of using AAHS control, to characterise the safety of the HPV virus-like particles, lacks clinical relevance. The clinically relevant question is what are the effects (benefits and harms) of qHPV vaccine—the whole product, not one of its components.

As crucial documents written before trials commence, it is concerning that study protocols did not include a rationale for the use of AAHS control. Protocols for V501-015, V501-019 and V501-020 stated: ‘To provide an appropriate control for the Quadrivalent HPV (Types 6, 11, 16, 18) L1 VLP Vaccine, the placebo used in this study will be Merck standard aluminum diluent (225 µg alum) in normal saline, USP (NaCl 0.9%)’. But the documents did not explain what made the adjuvant ‘appropriate’. We were also concerned that ClinicalTrials.gov entries described the control with a single word (‘placebo’, not even mentioning AAHS), and in one case did not even list the control.

The Consolidated Standards of Reporting Trials (CONSORT) guidelines for reporting RCTs call for study interventions for each group to be described ‘with sufficient details to allow replication, including how and when they were actually administered’.42 We think CONSORT should also ask investigators to document their rationale for selecting these interventions, particularly the choice of control. Such reporting would allow for more transparency and a more informed discussion around the appropriateness of chosen controls, and is consistent with regulatory advice to industry that ‘the choice of control group is always a critical decision in designing a clinical trial’.43

Limitations

Across all study documents, the only ingredients we found listed in the control were AAHS and normal saline (NaCl 0.9%). However, there may have been other ingredients. According to qHPV vaccine’s prescribing information, each dose of vaccine contains ‘9.56 mg of sodium chloride, 0.78 mg of L-histidine, 50 mcg of polysorbate 80, 35 mcg of sodium borate, <7 mcg yeast protein/dose and water’, in addition to AAHS and HPV virus-like particles.32–34 To test HPV virus-like particles, as the manufacturer stated was its intention in using an AAHS control, the control would logically have also included these other ingredients in addition to AAHS. We recommend that reports of trials (across publications, protocols, CSRs and registers) should contain a complete list of ingredients of all trial arms (if necessary, in supplementary materials when space limitations apply).

Our study may not represent a complete list of qHPV vaccine trial publications that under-report and misreport the control. In our screening, we identified a trial (V501-005) with identical problems to the trials included in this restoration; however, this trial was excluded from our analysis as it reported on the use of a monovalent experimental HPV vaccine which was never licensed.44

Conclusions

The sponsor’s stated rationale of using AAHS control in qHPV vaccine trials—to characterise the safety of the HPV virus-like particles—lacks clinical relevance, as the clinically relevant issue is to characterise the safety profile of the whole qHPV vaccine. The non-placebo control used in qHPV vaccine trials may have obscured an accurate assessment of safety of qHPV vaccine and the participant consent process of some trials raises ethical concerns.