Aluminium adjuvants versus placebo or no intervention in vaccine randomised clinical trials: a systematic review with meta-analysis and Trial Sequential Analysis.

OBJECTIVES: To assess the benefits and harms of aluminium adjuvants versus placebo or no intervention in randomised clinical trials in relation to human vaccine development.
DESIGN: Systematic review with meta-analysis and trial sequential analysis assessing the certainty of evidence with Grading of Recommendations Assessment, Development and Evaluation (GRADE). DATA SOURCES: We searched CENTRAL, MEDLINE, Embase, LILACS, BIOSIS, Science Citation Index Expanded and Conference Proceedings Citation Index-Science until 29 June 2021, and Chinese databases until September 2021. ELIGIBILITY CRITERIA: Randomised clinical trials irrespective of type, status and language of publication, with trial participants of any sex, age, ethnicity, diagnosis, comorbidity and country of residence. DATA EXTRACTION AND SYNTHESIS: Two independent reviewers extracted data and assessed risk of bias with Cochrane's RoB tool 1. Dichotomous data were analysed as risk ratios (RRs) and continuous data as mean differences. We explored both fixed-effect and random-effects models, with 95% CI. Heterogeneity was quantified with I2 statistic. We GRADE assessed the certainty of the evidence.
RESULTS: We included 102 randomised clinical trials (26 457 participants). Aluminium adjuvants versus placebo or no intervention may have no effect on serious adverse events (RR 1.18, 95% CI 0.97 to 1.43; very low certainty) and on all-cause mortality (RR 1.02, 95% CI 0.74 to 1.41; very low certainty). No trial reported on quality of life. Aluminium adjuvants versus placebo or no intervention may increase adverse events (RR 1.13, 95% CI 1.07 to 1.20; very low certainty). We found no or little evidence of a difference between aluminium adjuvants versus placebo or no intervention when assessing serology with geometric mean titres or concentrations or participants' seroprotection.
CONCLUSIONS: Based on evidence at very low certainty, we were unable to identify benefits of aluminium adjuvants, which may be associated with adverse events considered non-serious. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We seem to be the first to assess the benefits and harms of aluminium adjuvants by conducting a systematic review comparing aluminium adjuvants versus placebo or no intervention in combination with all types of vaccines.

We included 102 randomised clinical trials from a comprehensive search with no language limitations or restrictions on outcomes reported in the trials, type of aluminium adjuvant or type of vaccine.

The certainty of evidence is very low and this makes it difficult to draw firm conclusions.

Introduction

Vaccination is one of the major triumphs of modern medicine.1 2 Vaccination prevents infectious diseases, and the worldwide eradication of the deadly smallpox and the restriction of diseases such as poliovirus, measles and tetanus can largely be ascribed to the numerous successful mass vaccination programmes launched since the 1960s.1 2 Presently, COVID-19 vaccines are rolled out worldwide with speed to stop the COVID-19 pandemic.3 4 In addition to its intended effect, a vaccine may be accompanied by one or more harmful effects on administration. Harms may be considered non-serious (eg, mild, transient headache) or serious (eg, causing hospitalisation or death) and they may appear shortly after vaccine administration (eg, pain at the injection site) or belated (eg, autoimmune responses).

The human papilloma virus (HPV) vaccination programme was launched in the USA in 2006 in order to prevent HPV infection, one of the causes of cervical cancer and the second most common cancer in women.5 Despite that HPV vaccines have been assessed for efficacy (immunogenicity) in clinical trials, and approved based on their ability to raise a potent immune response against HPV and their ability to prevent persistent HPV infections,6 concerns have been raised about adverse events possibly related to the HPV vaccines formulation.7 8 Both the national vaccine adverse events reporting system in the USA and the European Union have received reports on a high number of adverse events suspected to be related to the HPV vaccination.8 However, no scientific evidence for an association was found.9 Several observational studies also failed to identify associations with clinical diagnoses.10–14 However, reasons to oppose these findings have been proposed.7 15 16

Vaccine toxicity, efficacy and effectiveness may originate from, or depend on a plethora of factors, including the vaccine components (eg, the antigen itself, the excipient or the adjuvant); interaction between different vaccine components; vaccine manufacture; overall vaccine composition; route of administration; dose; and number of booster vaccinations.17 Aluminium salts are widely used adjuvants, such as aluminium phosphate, aluminium hydroxide, aluminium potassium sulfate and amorphous aluminium hydroxyphosphate sulfate.18 They have been the standard adjuvants in vaccines against diphtheria, tetanus, and pertussis, haemophilus influenza type B, pneumococcus conjugates, hepatitis A, and hepatitis B.19 More recently, aluminium was coformulated with vaccines against HPV in the form of Adjuvant System 04 (aluminium hydroxide and monophosphoryl lipid A), aluminium hydroxide or amorphous aluminium hydroxyphosphate sulfate as well as in the worlds most used COVID-19 vaccines CoronaVac20 and Sinopharm Beijing Institute of Biological Products COVID-19 vaccine21 in the form of aluminium hydroxide.

The mechanism of action of aluminium, like for most adjuvants, is only partially understood. Its biological or physiological role is unknown. While aluminium is generally considered safe and is regularly ingested in food and water, it can be toxic based on the concentration, chemical form and the environment.22 Aluminium seems to have an impact on the immune system, which has rendered it useful as a vaccine adjuvant.19 23 Aluminium is believed to exert its adjuvant effects by stimulating Th2-type cell responses and antibody production through B cells activation,24 25 by activating the complement system, and by recruiting immune cells to the site of injection.24 26 27 At the injection site, aluminium promotes antigen uptake by specialised antigen-presenting immune cells, termed dendritic cells, as well as dendritic cell maturation.23 28 29 The consensus within the scientific community is that aluminium affects antigen uptake, induces danger signals, recruits various types of immune cells and elicits Th2 responses.30

One previous attempt to assess the effects of aluminium adjuvants with a review was undertaken in 2004 by Jefferson et al.31 The review covered existing evidence of adverse events to the aluminium-containing diphteria-tetanus-pertussis vaccine, but it did not assess benefits.31 Lin et al conducted the first meta-analysis on the efficacy of aluminium salts as an adjuvant for prepandemic influenza vaccines.32 Their results showed inferior seroprotection after aluminium-adjuvanted H5N1 vaccines compared with that conferred by non-adjuvanted counterparts; however, these findings only related to the prepandemic influenza vaccines. New adjuvants are being introduced continuously and the U.S. Food and Drug Administration (FDA) and the WHO do not require genotoxicity or cardiotoxicity studies of new aluminium adjuvants.33 The theory that aluminium adjuvant is responsible for symptoms following specific vaccine formulation is impossible to refute or prove based on the data from current clinical trials. For example, aluminium adjuvant has been administered to both the experimental and control groups in the vast majority of randomised clinical trials on HPV vaccines, thus masking aluminium adjuvant’s potentially harmful effects.34 Aluminium adjuvants, new or old, should be evaluated for benefits and harms on their own merits. While the consequences of adding aluminium to vaccines have been discussed broadly, no systematic review has been conducted to assess the effects of aluminium adjuvants across different types of vaccines.

The objectives of this review are to assess the benefits and harms of aluminium adjuvants vs placebo or no intervention in randomised clinical trials in relation to human vaccine development. Our aim was not to analyse the benefits and harms of vaccine formulations for prevention of a specific disease. The results of our systematic review could influence future vaccine formulation and bring on changes among policymakers and vaccine manufacturers to secure safe and efficient vaccines to people.

Methods

Detailed description of our methodology is in our prepublished protocol,35 PROSPERO protocol (CRD42017083013) and our online supplemental material.

Criteria for considering studies for this review

We searched for randomised clinical trials irrespective of type, status, date and language of publication. We included vaccine development trials comparing any type of aluminium adjuvant versus placebo or no intervention. We accepted any cointerventions of vaccines if planned to be delivered equally to the intervention groups. We used the trial results reported at maximum follow-up.

Types of outcome measures

Primary outcomes were serious adverse events,36 all-cause mortality and proportion of participants with the disease being vaccinated against. Secondary outcomes were health-related quality of life, non-serious adverse events and serological response.

Search methods for identification of studies

We searched the Cochrane Central Register of Controlled Trials (2021, Issue 7) in The Cochrane Library, MEDLINE Ovid (1946 to July 2021), Embase Ovid (1974 to July 2021), LILACS (Bireme; 1982 to July 2021), BIOSIS (Web of Science; 1969 to July 2021), Science Citation Index Expanded (Web of Science; 1900 to July 2021), and Conference Proceedings Citation Index-Science (Web of Science; 1990 to July 2021). In addition, we searched (September 2021) the Chinese Biomedical Literature Database (CBM), China Network Knowledge Information (CNKI), Chinese Science Journal Database (VIP) and Wanfang Database (online supplemental table S1). We also searched Google Scholar, The Turning Research into Practice (TRIP) Database, ClinicalTrials.gov (www.clinicaltrials.gov/), European Medicines Agency (EMA; www.ema.europa.eu/ema/), WHO International Clinical Trial Registry Platform (www.who.int/ictrp), The Food and Drug Administration (FDA; www.fda.gov) and pharmaceutical company sources for ongoing or unpublished trials (until March 2021). We applied EMA, FDA and several national medicines agencies (Australia, China, India, Japan, UK, Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Latvia, Liechtenstein, Lithuania, Luxembourg, Malta, Netherlands, Norway, Poland, Portugal, Romania, Slovakia, Slovenia, Spain and Sweden) for clinical study reports on trials fulfilling our inclusion criteria.

Data collection and analyses

Three review authors (SRK, SLK and MB) independently and in pairs screened titles and abstracts for inclusion of potentially eligible trials using Covidence (www.covidence.org). Following any unsolved disagreements, we asked a third author to arbitrate (JCJ or CG). The review author pair collected full-text trial reports/publications, and independently screened the full-texts and identified trials for inclusion. SRK extracted all data on all trials. SLK and MB each independently extracted half of the data. Extractions were compared and validated by SRK, SLK and MB and in case of disagreement, the same review authors consulted JCJ or CG.

Assessment of risk of bias in included studies

The review author pair (SRK, SLK and MB) independently assessed the risk of bias (RoB 1) of each included trial according to the recommendations in the Cochrane Handbook for Systematic Reviews of Interventions.37 We used the following bias risk domains: ‘allocation sequence generation’; ‘allocation concealment’; ‘blinding of participants and treatment providers’; ‘blinding of outcome assessment’; ‘incomplete outcome data’; ‘selective outcome reporting’ and ‘other bias’. We assessed the domains ‘blinding of outcome assessment’, ‘incomplete outcome data’ and ‘selective outcome reporting’ for each outcome. The trial was classified at overall ‘low risk of bias’ only if all the bias domains described in the previous paragraphs were classified at low risk of bias, or at ‘high risk of bias’ if any of the bias risk domains described above were classified at ‘unclear’ or ‘high risk of bias’.

We used the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system to assess the certainty of the body of evidence associated with each of the outcomes.38 We constructed a summary of findings table using the GRADEpro software.39 The GRADE system appraises the certainty of evidence based on the extent to which one can be confident that an estimate of effect or association reflects the item being assessed.

Measures of treatment effect and data synthesis

We visually investigated forest plots to assess the risk of statistical heterogeneity. We also assessed the presence of statistical heterogeneity using the χ2 test (threshold p<0.1) and measured the quantities of heterogeneity using the I2 statistic.40 41 We assessed reporting bias using funnel plots when 10 or more trials per comparison were included.

When the total proportion of participants experiencing any serious and non-serious adverse event was not reported, we extracted data from the highest proportion of participants experiencing an individual adverse event.

We performed subgroup analyses on (A) outcomes at low risk of bias compared with outcomes at high risk of bias or unclear risk of bias (collectively termed high risk of bias); (B) trials at low risk of vested interests compared with trials at high risks of vested interests;42 (C) according to aluminium adjuvants type; (D) according to different vaccines; (E) according to age groups; (F) according to different maximal follow-up periods; (G) according to participants’ health and (H) according to vaccines against extracellular or intracellular pathogens.

We assessed the potential impact of missing data with the ‘best-worst’ and ‘worst-best’ case scenarios.

Intervention effects were assessed with both random-effects model43 and fixed-effect model44 meta-analyses. The more conservative point estimate of the two (the analysis with the highest p value) was reported primarily. For analysis of the three primary outcomes, a p<0.025 was considered statistically significant45 because this would secure a familywise error rate below 0.05.

We analysed all primary and secondary outcomes using Trial Sequential Analysis (TSA) o.9.5.10 Beta software to control random errors.35

We included aluminium concentration (as described by trialist or manufacturer) as a covariate in meta-regression to assess whether the concentration influences the effect of aluminium adjuvant administration on outcomes.

Patient and public involvement

A patient and a representative of the public were involved in formulating the research question and the outcomes at the protocol stage. They were both involved in the interpretation and writing up of results. There are plans to disseminate the results of the research to the public and the relevant patient communities.

Results

Online supplemental figure S1 shows the flow of records obtained through electronic searches. We identified 15 958 records through database searching. We obtained 396 full-text reports that were assessed for eligibility. We excluded 280 records. We identified eight trials awaiting classification and six ongoing trials (online supplemental appendix 1).

We identified 102 randomised clinical trials including a total of 26 457 participants that fulfilled our inclusion criteria. Characteristics of included and excluded studies are given in online supplemental appendix 1. We were unable to identify any clinical study report from regulatory authorities that was eligible for inclusion in this review. We approached all corresponding authors to request missing information or explanations on unclear information and received some additional information from seven authors.

The 102 included trials were published between 1969 and 2021; 35 were conducted in the USA;46–80 13 in more than one country;81–94 6 in Canada;95–100 4 in China;101–104 4 in Belgium;105–108 4 in Africa;109–112 3 each in the UK,113–115 Taiwan116–118 and Australia;119–121 2 each in Thailand,122 123 Poland,124 125 Norway,126 127 Italy,128 129 Germany,92 130 Cuba,131 132 and Austria;133 134 and 1 each in Switzerland,135 Sweden,136 Singapore,137 Mali,138 Israel,139 India,140 France,141 Colombia,142 Chile and Bangladesh.143 Three trials did not report a country.144–146

Trial participants

The trials randomised different types of participants. Ninety trials randomised healthy participants; nine trials randomised participants with a disease diagnosis;50 57 60 71 90 91 119 129 134 and three trials did not describe the inclusion criteria of the participants.55 98 99

In regard of age, the trials randomised: infants (6 trials);86 113 125 132 143 145 children (11 trials);85 87 94 101 103 116 132 138 146–148 adolescents (2 trials);66 114 elderly (9 trials);50 60 62 83 97 100 111 119 139 and mixed populations (8 trials).57 78 91 102 104 108 112 123 Two trials did not specify the population type.55 134 The remaining 65 trials randomised adult participants.

Interventions and comparisons

Types of aluminium adjuvants

The included trials assessed different types of aluminium adjuvants: aluminium hydroxide (38 trials);49 50 57 59 67–70 72 74 81–83 88–90 92 93 96 101–103 105 106 110 112 114 118 122–124 126 133 134 140–142 144 aluminium phosphate (26 trials);46 47 53 60 62 63 66 75 80 86 87 97 108 109 111 115–117 121 131 132 136 139 145 146 149 alhydrogel (21 trials);48 51 52 54–56 58 73 76–79 98–100 107 113 120 125 127 137 amorphous aluminium hydroxyphosphate sulfate (2 trials);64 71 aluminium fluoride (1 trial);94 phosphate-treated aluminium hydroxide (1 trial);143 alhydrogel pretreated with phosphate buffer (1 trial);135 Adju-Phos (aluminium phosphate gel) (1 trial);95 aluminium potassium sulfate (1 trial);65 aluminium chloride (1 trial)61 and aluminium oxide (1 trial).150 Eight trials did not describe the type of aluminium adjuvant used.84 85 91 104 128–130 138

Vaccines against different viruses, bacteria, toxins or diseases

The included trials assessed the effects of vaccines against different viruses, bacteria, toxins or diseases: influenza (25 trials);49 50 54 56 67 68 84 88 98–100 102 104 115 117 118 121 126 130 133 137 139–141 150 Streptococcus pneumoniae (11 trials);74 85–87 94 108 111 135 143 145 146 respiratory syncytial virus (11 trials);60 62 63 79 80 82 96 97 105 119 144 human immunodeficiency virus (6 trials);47 59 75 110 128 129 Neisseria meningitidis (6 trials);53 61 66 109 127 138 Clostridium difficile (4 trials);57 69 78 83 dengue fever virus (4 trials);55 58 120 151 enterovirus (3 trials);101 103 116 Bacillus anthracis (3 trials);48 51 52 diphtheria and tetanus (2 trials);113 136 human papillomavirus (2 trials);65 72 Lyme borreliosis (2 trials);89 93 Haemophilus influenzae type B (2 trials);131 132 group B Streptococcus (2 trials);76 107 Staphylococcus aureus (2 trials);64 71 poliovirus (2 trials);124 125 Pseudomonas aeruginosa (2 trials);90 92 Alzheimer’s disease (2 trials);91 134 cytomegalovirus (2 trials);46 95 tetanus (1 trial);114 non-typeable Haemophilus influenzae (1 trial);107 Ross River virus (1 trial);81 hepatitis B (1 trial);112 malaria (1 trial);142 rabies and tetanus (1 trial);122 rabies (1 trial);123 Shigella flexneri (1 trial)77 and S. aureus and Candida albicans (1 trial).73

Overall, 41 trials assessed vaccines against extracellular pathogens (bacteria or toxins) and 61 trials assessed vaccines against intracellular pathogens (viruses).

Vaccine doses

The included trials administered different numbers of vaccine doses: 24% of the trials administered 1 dose;53 55 58 60 64 66 70 73 76 82 87 96 97 100 107 111 114 124 131 133 136 139 140 150 40% of the trials administered two doses;49–52 54 56 61–63 67 68 75 77 79 80 84 88 90 92 98 99 101–105 108 109 115–119 121 126 127 130 135 137 138 141 146 21% of the trials administered three doses;46 47 57 59 65 71 72 74 78 81 85 89 95 106 110 112 113 120 123 125 132 142 143 10% of the trials administered four doses;69 83 86 93 94 134 145 two trials administered five doses128 129 and one trial administered seven doses.91 Two trials did not specify the number of doses administered.122 144

Aluminium concentrations

The included trials used different aluminium concentrations ranging from 125 μg/dose to 6000 μg/dose.

Control groups

Two comparisons (from two trials) extracted in this review did not involve a vaccine (ie, comparison between saline placebo with or without aluminium).70 140 All the other control groups contained the same vaccine as the intervention group but without aluminium adjuvant.

Risk of bias within individual trials

Based on the information collected from the published reports and from authors, only 3/102 trials were at overall low risk of bias (all outcomes reported at low risk of bias). The remaining trials were at overall high risk of bias (online supplemental figure S2).

Effects of aluminium adjuvants

Serious adverse events

A total of 170/7627 (2.2%) participants who received aluminium adjuvants with or without vaccines suffered a serious adverse event vs 149/13 936 (1.1%) participants receiving no aluminium adjuvants with or without vaccines (risk ratio, RR 1.18, 95% CI 0.97 to 1.43; 21 563 participants; 62 trials; I2 0%; Bayes factor 548.28; very low certainty of evidence (figure 1, table 1, online supplemental figure S3). Visual inspection of the forest plot and I2 reveal no statistical heterogeneity. TSA showed that the cumulative Z-curve (blue full line with quadratic squares indicating each trial) touched the traditional boundary for harm. However, none of the trial sequential monitoring boundaries (etched curves above and below the traditional naive horizontal lines for statistical significance) were surpassed. The result is inconclusive as the required information size has not been achieved. The TSA-adjusted CI is 0.53 to 2.69 (Pc (proportion with an outcome in the control group) 1.0%, RR reduction or increase (RRR) 20%, alpha 2.5%, beta 10%, diversity 0%; diversity-adjusted required information size (DARIS) 110 696 participants) (online supplemental figure S4).

Figure 1

Meta-analysis of the effect of aluminium adjuvant compared with placebo or no intervention on the proportion of participants with one or more serious adverse events. M-H, Mantel-Haenszel.

Table 1

Summary of findings table

Summary of findings tableAluminium adjuvants compared with placebo or no intervention in vaccines
Patient or population: any populationSetting: any settingsIntervention: aluminium adjuvantsComparison: placebo or no intervention
Outcomes	Anticipated absolute effects* (95% CI)		Relative effect(95% CI)	No of participants(studies)	Certainty of the evidence(GRADE)	Comments
	Risk with placebo or no intervention	Risk with aluminium adjuvants
Proportion of participants with one or more serious adverse events	11 per 1000	12 per 1000(9 to 15)	RR 1.18(0.97 to 1.43)	21 563(62 RCTs)	⊕ΟΟΟVERY LOW †,‡,§	Aluminium adjuvants vs placebo or no adjuvants may have no effect on the proportion of participants with one or more serious adverse events, but the evidence was very uncertain. Imprecision and indirectness were considered ‘very serious’ and therefore downgraded twice
All-cause mortality	4 per 1000	4 per 1000(3 to 6)	RR 1.02(0.74 to 1.41)	21 886(63 RCTs)	⊕ΟΟΟVERY LOW ¶,**	Aluminium adjuvants vs placebo or no adjuvants may have no effect on all-cause mortality, but the evidence was very uncertain Imprecision was considered ‘very serious’ and therefore downgraded twice
Proportion of participants with one or more adverse events considered non-serious	338 per 1000	385 per 1000(362 to 409)	RR 1.13(1.07 to 1.20)	20 527(67 RCTs)	⊕ΟΟΟVERY LOW ††,‡‡,§§	Aluminium adjuvants may increase the proportion of participants with one or more adverse events considered non-serious but the evidence is very uncertain. Indirectness was considered ‘very serious’ and therefore downgraded twice.
Participants without seroprotection	118 per 1000	112 per 1000(91 to 139)	RR 0.95(0.77 to 1.18)	7845(14 RCTs)	⊕ΟΟΟVERY LOW ¶¶,***,†††	Aluminium adjuvants vs placebo or no intervention may have no effect on participants without seroprotection, but the evidence was very uncertain. Publication bias was considered ‘strongly suspected’ and therefore downgraded twice.
Health-related quality of life	0 per 1000	0 per 1000(0 to 0)	not estimable	(0 RCT)	–	No data
Participants with disease being vaccinated against	0 per 1000	0 per 1000(0 to 0)	not estimable	221(2 RCTs)	–	Too few data

GRADE Working Group grades of evidence.

High certainty: We are very confident that the true effect lies close to that of the estimate of the effect.

Moderate certainty: We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different.

Low certainty: Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect.

Very low certainty: We have very little confidence in the effect estimate: The true effect is likely to be substantially different from the estimate of effect.

*The risk in the intervention group (and its 95% CI) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI).

†Downgraded for risk of bias. Overall: 89% of the trials were at high risk of bias for this outcome and 11% of the trials were at low risk of bias for this outcome. Specifically, 17% of the trials reporting on serious adverse events were at high risk of bias in blinding of outcome assessors and 54% of the trials reporting on serious adverse events had unclear risk of bias in blinding of outcome assessors. Seventeen per cent of the trials reporting on serious adverse events were at high risk of bias in missing outcome data and 27% of the trials reporting on serious adverse events were at unclear risk of bias in missing outcome data. Four per cent of the trials reporting on serious adverse events were at high risk of bias in selective outcome reporting and 27% of the trials reporting on serious adverse events were at unclear risk of bias in selective outcome reporting.

‡Differences in outcomes measures: of the 61 trials that reported on serious adverse events, 8 trials reported only vaccine-related serious adverse events without giving information on the serious adverse events that potentially occurred but were not related to the vaccine. Other 14 trials reported that serious adverse events occurred but these were not described per intervention group because they were considered unrelated to the vaccine (a total of 107 serious adverse events reported to having occurred but not described per intervention group).

§Downgraded for imprecision. Optimal information size (n=110 696) not reached.

¶Downgraded for risk of bias. Overall: 79% of the trials were at high risk of bias for this outcome and 21% of the trials were at low risk of bias for this outcome. Specifically, 16% of the trials reporting on mortality had high risk of bias in blinding of outcome assessors and 43% of the trials reporting on mortality had unclear risk of bias in blinding of outcome assessors. Nine per cent of the trials reporting on mortality had high risk of bias in missing outcome data and 22% of the trials reporting on mortality had unclear risk of bias in missing outcome data. One per cent of the trials reporting on mortality had high risk of bias in selective outcome reporting and 21% of the trials reporting on mortality had unclear risk of bias in selective outcome reporting. However, the overall limitations were unlikely to influence this outcome.

**Downgraded for imprecision. Optimal information size (n=278 247) not reached.

††Downgraded for risk of bias. Overall: 75% of the trials were at high risk of bias for this outcome and 25% of the trials were at low risk of bias for this outcome. Specifically, 14% of the trials reporting on non-serious adverse events were at high risk of bias in blinding of outcome assessors and 49% of the trials reporting on serious adverse events were at unclear risk of bias in blinding of outcome assessors. Thirteen per cent of the trials reporting on non-serious adverse events were at high risk of bias in missing outcome data and 9% of the trials reporting on non-serious adverse events were at unclear risk of bias in missing outcome data. One per cent of the trials reporting on non-serious adverse events were at high risk of bias in selective outcome reporting and 26% of the trials reporting on serious adverse events were at unclear risk of bias in selective outcome reporting.

‡‡Downgraded for inconsistency. I2=85%. Visual inspection of funnel plot may suggest potential publication bias for smaller trials reporting a harmful effect of aluminium adjuvants in the placebo group. Regression-based Harbord test showed no small-study effects (beta=0.99)

§§Downgraded for indirectness. Differences in outcomes measures: a number of studies report only number of solicited or unsolicited adverse events, rather than total number of participants experiencing any adverse events. Out of the 66 trials reporting non-serious adverse events, only 34 trials reported the overall proportion of participants with one or more non-serious adverse event. From the remaining 32 out of the 66 trials reporting on non-serious adverse events we extracted data from the highest proportion of participants experiencing an individual adverse event.

¶¶Overall: 100% of the trials were at high risk of bias for this outcome. Specifically, 0% of the trials reporting on seroprotection were at high risk of bias in blinding of outcome assessors and 68% of the trials reporting on serious adverse events were at unclear risk of bias in blinding of outcome assessors. Seventy-four per cent of the trials reporting on seroprotection were at high risk of bias in missing outcome data and 0% of the trials reporting on serious adverse events were at unclear risk of bias in missing outcome data. Zero per cent of the trials reporting on seroprotection were at high risk of bias in selective outcome reporting and 23% of the trials reporting on serious adverse events were at unclear risk of bias in selective outcome reporting.

***Downgraded for inconsistency. I2=78%.

†††Visual inspection of funnel plot may suggest potential publication bias for smaller trials reporting an harmful effect of aluminium adjuvants in the intervention group. Regression-based Harbord test showed no small-study effects (beta=0.99).

GRADE, Grading of Recommendations Assessment, Development, and Evaluation; RR, risk ratio.

Subgroup analyses

Test for subgroup differences showed no difference when comparing the effects of aluminium adjuvants in trials at high risk of bias to trials at low risk of bias; in trials at risk of vested interest to trials at low risk of vested interest; trials according to different aluminium types; trials with different vaccines; trials with different participants’ ages; trials with different follow-up durations; trials with participants with different diagnoses compared with healthy participants; and trials assessing vaccines against different pathogens types (online supplemental Figures S5–S12).

Sensitivity analyses

A total of 21/7648 (0.3%) participants in the intervention group vs 18/13 954 (0.1%) participants in the control group were lost to follow-up. Incomplete outcome data alone seemed to have the potential to influence the result in the ‘worst-best’ case scenario analysis (online supplemental figure S13). The ‘best-worst’ case scenario analysis showed that incomplete outcome data did not have the potential to influence the result (online supplemental figure S14).

Meta-regression showed that the proportion of participants with serious adverse events was not affected by the aluminium concentration used in the vaccine (p=0.28).

Due to several trials with zero events, we performed meta-analysis also with OR. The results did not change (online supplemental figure S15).

Individual serious adverse events analyses

Meta-analyses showed no evidence of a difference between aluminium adjuvants vs control when assessing individual serious adverse events (online supplemental analysis S1). Individual serious adverse events reported only in one trial that were not possible to meta-analyse are shown in online supplemental table S2.

All-cause mortality

A total of 61/7782 (0.8%) aluminium participants died compared with 57/14 104 (0.4%) control participants (RR 1.02, 95% CI 0.74 to 1.41; 21 886 participants; 63 trials; I2 0%; Bayes factor 2.96; very low certainty evidence (figure 2, table 1, online supplemental figure S16). Visual inspection of the forest plot and I2 indicated no statistical heterogeneity. Funnel plot showed no publication bias. TSA showed that the accrued information for all-cause mortality was below 5% of the DARIS (Pc 0.4%, RRR 20%, alpha 2.5%, beta 10%, diversity 0%; DARIS 278 247).

Figure 2

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on all-cause mortality. M-H, Mantel-Haenszel.

Test for subgroup differences showed no difference when comparing trials at high risk of bias to trials at low risk of bias; trials at risk of vested interest to trials at low risk of vested interest; trials with no vaccine cointervention to trials with vaccine cointervention; trials with different aluminium types; trials with different vaccines; trials with different participants’ ages; trials with different participants’ diagnoses; and trial assessing vaccines against different pathogens types (online supplemental figures S17–S23). Due to lack of relevant data, it was not possible to conduct the subgroup analyses on trials with different follow-up durations.

A total of 28/7909 (0.35%) participants in the aluminium group vs 20/14 173 (0.14%) participants in the control group were lost to follow-up. Incomplete outcome data alone seemed to have the potential to influence our result in the ‘worst-best’ case scenario showing a harmful effect of aluminium adjuvants compared with placebo (online supplemental figure S24). The ‘best-worst’ case analysis showed that incomplete outcome data did not have the potential to influence our result (online supplemental figure S25).

Meta-regression showed that the proportion of participants with all-cause mortality was not affected by the aluminium concentration used in the vaccine (p=0.88).

Due to several trials with zero events, we performed meta-analysis also with OR. The results did not change (online supplemental figure S26).

Participants with disease

Only two trials (one event) reported on the proportion of participants that developed the disease they were vaccinated against (online supplemental figure S27).

Adverse events considered non-serious

Out of the 67 trials reporting adverse events considered non-serious, 34 trials reported the overall proportion of participants with one or more adverse events considered non-serious. From the remaining 33 trials reporting on adverse events considered non-serious, we extracted data from the highest proportion of participants experiencing an individual adverse event.

A total of 3760/7098 (52.9%) aluminium participants experienced one or more non-serious adverse events compared with 4537/13 429 (33.8%) in control participants. Meta-analysis of these trials showed that aluminium adjuvants compared with placebo or no intervention may increase the proportion of participants with one or more adverse events considered non-serious (RR 1.13, 95% CI 1.07 to 1.20; participants=20 527; trials=67; I2=85%; Bayes factor 3.02E+26; the evidence was very uncertain (figure 3, table 1, online supplemental figure S28). TSA of non-serious adverse events shows that the cumulative Z curve crosses the boundary for harm, indicating that there was enough information to confirm that aluminium adjuvants compared with placebo or no intervention increases the risk of one or more non-serious adverse events (TSA Pc 33.5%, RRR 20%, alpha 2.5%, beta 10%, diversity 78%; DARIS 28 384. TSA adjusted CI 1.06 to 1.23 (online supplemental figure S29).

Figure 3

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on the proportion of participants with one or more non-serious adverse events. M-H, Mantel-Haenszel.

Visual inspection of the funnel plot and regression-based Harbord test showed no publication bias or small-study effects (beta=0.99).

Test for subgroup differences was statistically significant in the subgroup analysis according to vaccine type (p<0.00001; online supplemental figure S30) and age (p=0.007; online supplemental figure S31).

Test for subgroup differences showed no difference when comparing the effect of aluminium adjuvants in trials at low risk of bias to trials at high risk of bias; in trials at low risk of vested interest to trials at risk of vested interest; in trials with different aluminium salts; in trials with different follow-up durations; in trials with participants with different health status; and trials assessing vaccines against different pathogen types (online supplemental figures S32–S37).

A total of 195/7392 (2.6%) participants in the aluminium group vs 186/13 341 (1.4%) participants in the control group were lost to follow-up. Incomplete outcome data did not have the potential to influence our results.

We included aluminium concentration (as described by trialists) as a covariate in meta-regression to assess whether aluminium concentration has an impact on the effect sizes of the proportion of participants with adverse events considered non-serious. Meta-regression showed that the proportion of participants with adverse events considered non-serious was not affected by the aluminium concentration used in the vaccine (p=0.68)

Individual non-serious adverse events

We performed meta-analysis on each of the 145 reported individual adverse event considered non-serious. Mainly, local injection site reactions were increased in the aluminium group (online supplemental analysis S2 and table S3).

Serological response

Serological response was assessed by different analytical assays and was reported as either geometric mean titre (GMT, 31 trials) or geometric mean concentration (GMC, 11 trials).

Meta-analyses showed no or little evidence of a difference between aluminium adjuvants vs placebo or no intervention when assessing GMT or GMC (figures 4 and 5, online supplemental figures S38 and S39).

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on the geometric mean titres grouped by analytical assay. IV, inverse variance.

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on the geometric mean concentrations grouped by analytical assay. IV, inverse variance.

Subgroup and sensitivity analyses for serology

Subgroup and sensitivity analysis for the serological response is reported in online supplemental figures S40–S43.

Seroprotection

Meta-analysis showed that there was no evidence of a difference between aluminium adjuvants compared with placebo or no intervention when assessing seroprotection (RR 0.95, 95% CI 0.77 to 1.18; trials=14; I2 78%. Bayes factor 3.11; low certainty of evidence) (figure 6, online supplemental figure S44). Visual inspection of the forest plot and I2 statistics indicated high heterogeneity (I2 78%).

Figure 6

Meta-analysis of the effect of aluminium adjuvants compared with placebo or no intervention on seroprotection. M-H, Mantel-Haenszel.

Subgroup and sensitivity analyses for seroprotection

Subgroup and sensitivity analyses for seroprotection is reported in online supplemental text and figure S45 and S46.

Discussion

This review included 102 randomised clinical trials assessing a total of 26 457 participants. Aluminium adjuvants versus placebo or no adjuvants may have no effect on the proportion of participants with one or more serious adverse events and on all-cause mortality, but the evidence was very uncertain. Two trials reported on the proportion of participants with the disease they were vaccinated against. However, only one event was reported. None of the trials reported on quality of life. Aluminium adjuvants versus placebo or no adjuvants seem to increase the proportion of participants with one or more adverse events considered non-serious, but the evidence was very uncertain. We found no or little evidence of a difference between aluminium adjuvants vs placebo or no intervention when assessing geometric mean titres or concentrations. Aluminium adjuvants versus placebo or no intervention may have no effect on participants without seroprotection, but the evidence was very uncertain.

Strengths and weaknesses

We seem to be the first to conduct a systematic review comparing aluminium adjuvants versus placebo or no intervention in any type of vaccine. We followed our peer-reviewed protocol which was published before the literature search began,35 and we conducted the review using the methods recommended by Cochrane.37 152 We reported our review according to the PRISMA statement153 (online supplemental table S4).

Our systematic review has several limitations. Despite our inclusion criteria being broad, we could only find phase I or II trials that met our inclusion criteria. This limitation is because phase III or IV trials of marketed vaccines are mainly designed with an active comparator (another vaccine or alleged ‘placebo’ with aluminium), and therefore, these trial designs did not match the inclusion criteria of our review.

Another limitation of the applicability of our results is that we chose maximum follow-up as our time point of primary interest. This approach does not allow us to make conclusions on the effect of aluminium adjuvants on safety and immunogenicity after each individual vaccine dose (for those trials having multiple vaccine injections).

Another limitation is that we identified high clinical heterogeneity, especially within the immunogenicity outcome. Included trials did not use the same assays to assess the serological response and most of the trials had multiple assays performed. We chose to analyse only the geometric mean titre or concentration reflecting the data from the first assay presented in the publication; however, we are aware that this might limit the strength of our findings.

We chose to merge multiple groups in those trials that used vaccines with different antigen concentrations. In so doing, we were unable to conclude whether the effect of aluminium adjuvants is correlated to the effect of different antigen concentrations.

Only two trials (one event in total) reported on the proportion of participants with disease they were vaccinated against and none of the authors provided us with such data when contacted by email. Therefore, our conclusion regarding the effect of aluminium adjuvants on the immunogenicity is based on the surrogate outcome of the serological response to vaccine measured by different assays and on the seroprotection values as defined by the trialists.

Of the 62 trials that reported on serious adverse events, 8 trials reported only vaccine-related serious adverse events.49 51 81 100 104 107 108 115 Of the 62 trials that reported on serious adverse events, 14 trials reported that serious adverse events occurred but these were not assigned per intervention group because they were considered unrelated to the vaccine (a total of 107 serious adverse events reported to having occurred but not described per intervention group).50 66–68 77 83 84 86 91 97 118 121 134 146 Only 7/102 authors contacted provided us with all or some of the data requested (see characteristics of included studies in the online supplemental appendix 1).

Out of the 67 trials that reported on adverse events considered non-serious, 34 trials reported the overall proportion of participants with one or more adverse event considered non-serious.54 56 58 64 66 69 70 72 77–79 82 83 85 87 90 92 95 97 102 103 105–107 110 113 117 124 126 133 137 139 145 149 154 From the remaining 33/67 trials reporting on adverse events considered non-serious, we extracted data from the highest proportion of participants experiencing an individual adverse event.46 49–51 53 62 76 80 81 84 86 91 93 100 101 109 111 114 115 118 120 121 124 125 128 129 135 136 140–143 150 A substantial number of trials reported only solicited adverse events instead of solicited and unsolicited adverse events as a combined outcome. This limitation may have resulted in an underestimation of the unsolicited adverse events that might have occurred but were not reported.

Agreements and disagreements with other studies or reviews

Jefferson et al reviewed evidence of adverse events after exposure to aluminium-containing vaccines against diphtheria, tetanus and pertussis, alone or in combination, compared with identical vaccines, either without aluminium or containing aluminium in different concentrations.31 They included three randomised trials, four semirandomised trials, and one cohort study. They found that in young children, vaccines with aluminium hydroxide caused significantly more erythema and induration than plain vaccines and significantly fewer reactions of all types. In older children, there was an association with local pain lasting up to 14 days. Despite a lack of good-quality evidence, the authors surprisingly recommend against any further research on this topic.

Lin et al conducted the first meta-analysis on the efficacy of aluminium salts as an adjuvant for prepandemic influenza vaccines.32 They included a total of nine randomised clinical trials (published during 2006–2013), including 22 comparisons in 2467 participants that compared aluminium-adjuvanted H5N1 vaccines versus non-adjuvanted counterparts.32 Their results showed an inferior seroprotection after aluminium-adjuvanted H5N1 vaccines compared with that conferred by non-adjuvanted counterparts. Furthermore, H5N1 vaccines with aluminium adjuvants were associated with a significantly higher risk of pain/tenderness at the injection site during the 7 days after the first vaccination and after the second dose vs the non-adjuvanted counterparts.

Jørgensen et al set out to assess the benefits and harms of the HPV vaccines in clinical study reports obtained from the European Medicines Agency and GlaxoSmithKline from 2014 to 2017.155 They included 24 randomised clinical trials comparing an aluminium-adjuvanted HPV vaccine vs a placebo or active comparator in healthy participants of all ages. They found that at four years follow-up, the HPV vaccines decreased HPV-related precursors to cervical cancer and treatment procedures but increased serious nervous system disorders (exploratory analysis) and general harms.131 As the trials included in their review were primarily designed to assess benefits and not adequately designed to assess harms, the extent to which the benefits outweigh the harms was unclear.

In agreement with Lin et al,32 our systematic review does not find an increased serological response of aluminium-adjuvanted vaccines compared with that conferred by non-adjuvanted counterparts. Also, in agreement with both Jefferson et al31 and Lin et al32 we find an increase in local injection site reactions after administration of aluminium-adjuvanted vaccines.

Implications for practice and research

Considering the lack of good-quality evidence to assess beneficial and harmful effects of adding aluminium to vaccines as presented here, relevance of this adjuvant should be investigated in future studies. Questions on aluminium form, concentration and size remain unanswered due to scarcity or lack of data. Questions on the effects of aluminium adjuvants on vaccine effectiveness also remain unanswered.

Future randomised clinical trials in humans should be conducted according to the ethical principles that have their origin in the Declaration of Helsinki, and that are consistent with the International Council for Harmonisation Good Clinical Practice guidelines and the applicable regulatory requirement(s).156 157 Such trials should be designed in accordance with guidelines for clinical trials (Standard Protocol Items: Recommendations for Interventional Trials)158 and reported in accordance with the Consolidated Standards of Reporting Trials.159