Efficacy of approved vaccines to prevent COVID-19: a systematic review and network meta-analysis of reconstructed individual patient data from randomized trials.

Aim: To optimize vaccination strategy, evidence on vaccine efficacy against COVID-19 is needed. Method: The present network meta-analysis uses reconstructed individual patient data from phase III trials on vaccine efficacy (VE), identified through MEDLINE, EMBASE, and Cochrane library (CENTRAL) peer-reviewed and published in English before August 31, 2021. The primary outcome was the VE against confirmed COVID-19 at any time after the first dose as defined in each study. VE was re-estimated using the two-stage approach. Poisson regression models were applied to each trial at the first stage, and the incidence risk ratio (IRR) and their 95% CI were aggregated to allow random-effects network meta-analysis (NMA) at the second stage. VE was expressed as: (1-IRR) × 100. The study protocol is registered in PROSPERO (CRD42020200012).
Results: A total of eight studies, evaluating nine different vaccines were identified and analyzed. Between April 23, 2020 and January 05, 2021, 210,418 participants were recruited in 354 sites worldwide. During a median (IQR) follow-up duration of 69.8 (69.7-70.3) days, 2131 confirmed COVID-19 cases occurred (604; 26.0 per 1000 person-years in vaccine recipients and 1527; 85.9 per 1000 person-years in the control group). The mRNA-1273 vaccine was the most effective (P-score 0.99); at any time after dose 1, incidence reduction for mRNA-1273 ranged from 78% to 98% compared to the other vaccines.
Conclusion: Our results provide evidence for the short-term superiority of mRNA vaccines, especially the mRNA-1273 vaccine in prevention of COVID-19 in different populations. Supplementary Information: The online version contains supplementary material available at 10.1007/s10389-022-01707-1.

Introduction

Mass vaccination campaigns significantly contribute to containing the COVID-19 pandemic. Despite the high circulation of variants of concern, the efficacy of the current vaccines approved worldwide for emergency use authorization (EUA) against COVID-19 in the general population (Dagan et al. 2021; Jara et al. 2021; Kissling et al. 2021) remains similar to that reported in clinical trials (Baden et al. 2021; Polack et al. 2020; Voysey et al. 2021; Logunov et al. 2021; Sadoff et al. 2021; Al Kaabi et al. 2021; Heath et al. 2021; Tanriover et al. 2021). To optimize the vaccination strategy and guide public health recommendations, evidence on the most effective and safe vaccine against COVID-19 is needed. Previous meta-analysis on COVID-19 vaccine efficacy had focused on aggregate data, which were limited to evaluate the dynamic over time of the vaccine efficacy (Harder et al. 2021; Sharif et al. 2021). To address this issue, we initiated in July 2020 a network meta-analysis protocol (PROSPERO registration: CRD42020200012) to compare and rank the efficacy of COVID-19 vaccines using a reconstructed individual patient data (IPD) from published Kaplan–Meier curves due to heterogeneity of time-point of analysis and statistical methods. Here, we report preliminary results on comparison and ranking efficacies of vaccines against COVID-19.

Methods

Search strategy and selection criteria

We searched through MEDLINE, EMBASE, and Cochrane library (CENTRAL) peer-reviewed phase 3 randomized controlled trials (RCTs) that investigated COVID-19 vaccine efficacy, published in English before August 31, 2021. We included RCTs that compared efficacy against any confirmed cases of COVID-19 using reverse transcriptase polymerase chain reaction (O) at different time-points after the first dose (T), any candidate vaccine approved worldwide for EUA to prevent COVID-19 (I), in healthy adults or patients at high risk for SARS-CoV-2 infection (P), versus placebo or vaccine other than SARS-CoV-2 (C). Randomized studies in which the Kaplan–Meier plot did not report the number of at-risk participants were excluded. Two authors (AD and MCB) identified relevant studies, independently reviewed full texts, and disagreements were resolved by discussion. Data were extracted as described in the PROSPERO protocol, and risk of bias was assessed using the Cochrane Risk of Bias tool-2 (Rob-2).

Vaccine exposure

We considered any candidate vaccine approved worldwide for EUA to prevent COVID-19: BNT162b2 manufactured by Pfizer/BioNTech, is a lipid nanoparticle-formulated, nucleoside-modified RNA encoding the SARS-CoV-2 full-length spike; 30 μg; ChAdOx1 nCoV-19 [AZS1222]: manufactured by AstraZeneca, is a recombinant-deficient chimpanzee adenoviral vector containing the SARS-CoV-2 structural glycoprotein antigen: spike protein; nCoV-19; 2.2–6.5×1010 viral particle (VP); mRNA-1273: manufactured by MODERNA, is a lipid nanoparticle (LNP)-encapsulated modified RNA encoding the perfusion stabilized full-length spike protein of the SARS-CoV-2 virus; 100 μg; WIV04 (5 μg) and HB02 (4 μg): manufactured by Siopharm, are inactivated SARS-CoV-2 strains created from Vero cells with aluminum hydroxide adjuvant; Gam-COVID-Vac: manufactured by the Moscow City Health Department, Russian Direct Investment Fund, Sberbank, and RUSAL, is heterologous prime-boost which combined two vector vaccines based on rAd type 26 (rAd26) and rAd type 5 (rAd5) carrying the gene for SARS-CoV-2 full-length glycoprotein S; Ad26.COV2.S: manufactured by Janssen/Johnson & Johnson, is a replication-incompetent adenovirus type 26 (Ad26) vectored vaccine encoding a stabilized variant of the SARS-CoV-2 S protein (5×1010 VP); NVX-CoV2373: manufactured by Novavax, is a recombinant nanoparticle encoding the full-length spike glycoprotein of the prototype strain plus Matrix-M adjuvant (5 μg of NVX-CoV2373 plus 50 μg of Matrix-M adjuvant); CoronaVac: manufactured by the Turkish Health Institutes Association/Sinovac Research & Development, is inactivated whole-virion SARS-CoV-2 vaccine (3 μg of SARS-CoV-2 virion plus 0.45 mg/ml of aluminum hydroxide).

Outcomes and data synthesis

Vaccine efficacy (VE) against confirmed COVID-19 at any time after the first dose as defined in each study was the primary outcome. Secondary outcomes were VE at different time-points: (i) from randomization to day-21 after dose 1 and (ii) starting 7 days after dose 2. IPD were reconstructed by scanning the published Kaplan–Meier cumulative incidence curves using the WebPlotDigitizer software (Rohatgi 2021), then applying the reconstruction algorithm of Guyot and Colleagues (2012), which uses the magnitudes and locations of steps in the Kaplan–Meier curves, together with the numbers of patients at-risk, to infer the number of events and censorings occurring within each time interval. VE was re-estimated using the one-stage approach using the mixed Cox regression models with trials random-effects to account for difference in the study design and the background risk of COVID-19 during study. VE was expressed as: (1-incidence risk ratio [IRR]) × 100. To choose the preferred regimen, the P-score ranging from 0 (worse vaccine) to 1 (best vaccine) was computed for each vaccine, then the vaccine with a higher P-score was selected as better than each competing vaccine. Heterogeneity and inconsistency were quantified using the global Q test proposed by Rucker (Schwarzer et al. 2015). The Q statistic is the sum of statistic for heterogeneity, which represent the proportion of total variation in study estimates (within-designs), and a statistic for inconsistency (between-designs), which represents the variability of vaccine effect between direct and indirect comparisons at the meta-analytic level. To visualize and identify the nodes of single-design inconsistency, we used a network heat plot. Consistency between direct and indirect comparisons was checked using the so-called node-splitting. Sensitivity analysis was conducted by grouping vaccines according to their type (mRNA, viral vector, inactivated and recombinant protein).

Results

Study characteristics and risk of bias

Of 666 retrieved citations, 52 were full-text reviewed, and 8 were included in the quantitative analysis (Fig. 1) (Baden et al. 2021; Polack et al. 2020; Voysey et al. 2021; Logunov et al. 2021; Sadoff et al. 2021; Al Kaabi et al. 2021; Heath et al. 2021; Tanriover et al. 2021). Figure 2 shows the network for efficacy captured by the SARS-CoV-2 vaccines. Reconstructed IPD are shown in Fig. S1; they agree exactly with reported data for each vaccine groups and for each trial. Between April 23, 2020 and January 05, 2021, 210,418 participants were recruited in 354 sites worldwide. Of these participants, 124,099 (59%) were male with a median age ranging from 36.1 to 56 years, 81,521 (38.7%) had a comorbidity, including hypertension, diabetes and obesity, and 8401 (4%) have had a positive PCR or IgG at baseline. During a median (interval inter quartile [IQR]) follow-up duration of 69.8 (69.7–70.3) days, 2131 confirmed COVID-19 cases occurred (604; 26.0 per 1000 person–years in vaccines recipient and 1527; 85.9 per 1000 person–years in the control group). Figure 3 shows the vaccines efficacy compared with controls at different time-points after dose 1. A risk of attrition bias (incomplete outcome data) was detected in some trials (Fig. S2). Furthermore, no evidence of the presence of publication bias was detected (Fig. S3).

Fig. 1

PRISMA Flowchart of studies selected for meta-analysis of RCT COVID-19 vaccines. RCT: randomized clinical trial

Fig. 2

Network graph of eligible COVID-19 vaccines comparisons for efficacy. Line width is proportional to the number of trials comparing every pair of vaccine. The size of the circle is proportional to the number of participants assigned to receive the vaccine; BNT162b2 (lipid nanoparticle-formulated, nucleoside-modified RNA encoding the SARS-CoV-2 full-length spike): 30 μg; ChAdOx1 nCoV-19 (AZS1222): recombinant-deficient chimpanzee adenoviral vector containing the SARS-CoV-2 structural glycoprotein antigen (spike protein; nCoV-19): 2.2–6.5×1010 viral particle (VP); mRNA-1273 lipid nanoparticle (LNP)-encapsulated modified RNA encoding the perfusion stabilized full-length spike protein of the SARS-CoV-2 virus): 100 μg; WIV04 (5 μg) and HB02 (4 μg): inactivated SARS-CoV-2 strains created from Vero cells with aluminum hydroxide adjuvant; Gam-COVID-Vac: heterologous prime-boost which combined two vector vaccine based on rAd type 26 (rAd26) and rAd type 5 (rAd5) carrying the gene for SARS-CoV-2 full-length glycoprotein S; Ad26.COV2.S: replication-incompetent adenovirus type 26 (Ad26) vectored vaccine encoding a stabilized variant of the SARS-CoV-2 S protein (5×1010 VP); NVX-CoV2373: recombinant nanoparticle encoding the full-length spike glycoprotein of the prototype strain plus Matrix-M adjuvant (5 μg of NVX-CoV2373 plus 50 μg of Matrix-M adjuvant); CoronaVac: inactivated whole-virion SARS-CoV-2 vaccine (3 μg of SARS-CoV-2 virion plus 0.45 mg/ml of aluminum hydroxide)

Fig. 3

Efficacy of vaccines against COVID-19 at different time-points compared with control from reconstructed individual patient data. Vaccine efficacy estimates are provided as 1 minus incidence risk ratio (IRR) expressed as percentage with 95% confidence interval. BNT162b2 (lipid nanoparticle-formulated, nucleoside-modified RNA encoding the SARS-CoV-2 full-length spike): 30 μg; ChAdOx1 nCoV-19 (AZS1222): recombinant-deficient chimpanzee adenoviral vector containing the SARS-CoV-2 structural glycoprotein antigen (spike protein; nCoV-19): 2.2–6.5×1010 viral particle (VP); mRNA-1273 lipid nanoparticle (LNP)-encapsulated modified RNA encoding the perfusion stabilized full-length spike protein of the SARS-CoV-2 virus): 100 μg; WIV04 (5 μg) and HB02 (4 μg): inactivated SARS-CoV-2 strains created from Vero cells with aluminum hydroxide adjuvant; Gam-COVID-Vac: heterologous prime-boost which combined two vector vaccine based on rAd type 26 (rAd26) and rAd type 5 (rAd5) carrying the gene for SARS-CoV-2 full-length glycoprotein S; Ad26.COV2.S: replication-incompetent adenovirus type 26 (Ad26) vectored vaccine encoding a stabilized variant of the SARS-CoV-2 S protein (5×1010 VP); NVX-CoV2373: recombinant nanoparticle encoding the full-length spike glycoprotein of the prototype strain plus Matrix-M adjuvant (5 μg of NVX-CoV2373 plus 50 μg of Matrix-M adjuvant); CoronaVac: inactivated whole-virion SARS-CoV-2 vaccine (3 μg of SARS-CoV-2 virion plus 0.45 mg/ml of aluminum hydroxide)

At any time after the first dose

mRNA-1273 was the most effective vaccine to reduce incident cases of COVID-19 with a probability of 99.9% (P-score 0.999). Incidence reductions were 61% (95% CI, 33–78%) compared with BNT162b2 (P-score 0.881). The corresponding incidence reductions were 75% (56–85%), 76% (58–87%), 76% (56–87%), 79% (63–88%), 84% (74–90%), and 84% (75–92%) compared with Sputnik V (Gam-COVID-Vac; P-score 0.672), NVX-CoV2373 (P-score 0.617), ChAdOx1 nCov-19 (P-score 0.616), HB02 (P-score 0.521), Ad26.COV2.S (P-score 0.298), and WIV04 (P-score 0.231) vaccines, respectively. Incidence reductions were 87% (77–93%) for mRNA-1273, 67% (48–79%) for BNT162b2, 49% (21–67%) for Sputnik V, 46% (13–66%) for NVX-CoV2373, 46% (9–68%) for ChAdOx1 nCov-19, and 38% (2–61%) for HB02 vaccine compared with CoronaVac (P-score 0.164) recipient (Table S1).

Between randomization to 21-day after dose 1

Sputnik V was the most effective vaccine (P-score 0.937) followed by the ChAdOx1 nCov-19 vaccine (P-score 0.751). Compared with the WIV04 vaccine (P-score 0.529), COVID-19 incidence reduction for Sputnik V was 51% (5–75%). The corresponding incidence reductions were 60% (32–76%), 63% (36–79%), and 66% (21–85%) compared with CoronaVac (P-score 0.388), NVX-CoV2373 (P-score 0.305), and mRNA-1273 vaccines (P-score 0.272), respectively. Incidence reductions were 52% (8–75%) for HB02 (P-score 0.669), 56% (33–71%) for BNT162b2 (P-score 0.740), and 71% (59–80%) for Sputnik V compared with a single dose of Ad26.COV2.S vaccine (P-score 0.072) (Table S2).

Starting 7 days after the second dose

One week after the second dose or 35 days after the single dose of Ad26.COV2.S, the mRNA-1273 vaccine remained the most effective (P-score 0.929) with incidence reductions from 68% to 95% when compared with CoronaVac (P-score 0.574) and other vaccines. The corresponding incidence reductions were 73% to 95% for BNT162b2 (P-score 0.913), and 60% to 92% for NVX-CoV2373 (P-score 0.789) when compared with Sputnik V (P-score 0.463), HB02 (P-score 0.457), ChAdOx1 nCov-19 (P-score 0.402), WIV04 (P-score 0.278), and Ad26.COV2.S (P-score 0.196) vaccines, respectively (Table S3) (Table 1).

Table 1

Characteristics of included vaccination trials investigating the efficacy of COVID-19

Authors (NCT number), Design and enrollment period	Country (number of study sites)	Population (age, range)	Sample (% of men)	Vaccines: type, dosing information, and schedule (number of participants)	Control (number of participants)	Storage, conditions	Number of dose and route of administration	Primary outcomes	Sponsorship	Overall risk of bias
Polack, 2020 (NCT04368728), Phase 2–3randomized, observer-blind, placebo-controlled trial; July 27, 2020 to November 14, 2020	US (130 sites), Argentina (1 site), Brazil (2 sites), South Africa (4 sites), Germany (6 sites), and Turkey (9 sites)	Healthy or stable chronic disease condition: HIV, HBV, HCV (16 years or older)	43,448 (50.6)	BNT162b2 (lipid nanoparticle-formulated, nucleoside-modified RNA encoding the SARS-CoV-2 full-length spike): 30 μg (n=21,720)	Placebo (sterile saline, 0.9% sodium chloride n=21,728)	-80° to -60°C; 2-8°C for 30 days; room temperature ≤ 2h	Two doses 21 days apart in intramuscular in the deltoid muscle	Efficacy of BNT162b2 against confirmed COVID-19 with onset at least 7 days after the second dose in participants who had been without serologic or virologic evidence of SARS-CoV-2 infection up to 7 days after the second dose.	BioNTech and Pfizer	High
Voysey, 2020 (NCT04400838, NCT04444674), Phase 2–3 randomized, single-blind (COV002, UK), Phase 3 randomized, single-blind (COV003, Brazil); April 23, 2020 to November 4, 2020	UK (19 sites), Brazil (6 sites)	Professions with high risk exposure to virus (Health or social care workers, 18 years or older)	11,636 (39.5)	ChAdOx1 nCoV-19 (AZS1222): recombinant-deficient chimpanzee adenoviral vector containing the SARS-CoV-2 structural glycoprotein antigen (spike protein; nCoV-19) COV002: (two doses) LD/SD Low dose (LD): 2.2 ×1010 VP followed by a Standard-dose (SD): 3.5–6.5×1010 VP; n=1367) Two standards doses: SD/SD (n=2277) COV003 (two doses) Standard-dose (3.5–6.5×1010 VP; n=2063)	COV002 (LD/SD): Meningococcal group A, C, W, and Y conjugate vaccine (MenACWY; n=1374) COV002 (SD/SD): MenACWY (n=2430) COV003: MenACWY for the first dose and saline for the second dose (n=2,025)	2-8°C for 6 months;	Two doses 21 days apart in intramuscular in the deltoid muscle	Efficacy of ChAdOx1 nCoV-19 against virologically confirmed, symptomatic COVID-19, defined as a nucleic acid amplification test-positive swab (NAAT) positive swab combined with at least one qualifying symptom (fever ≥ 37.8°C, cough, shortness of breath, anosmia, or aguesia) in seronegative participants more than 14 days after a second dose.	UK Research and Innovation, National Institute of Health Research (NIHR), Coalition for Epidemic Preparedness Innovations, NIHR Oxford Biomedical Research Centre, Thames Valley and South Midlands NIHR Clinical Research Network, and AstraZeneca	High
Baden, 2020 (NCT04470427), Phase 3 randomized, observer-blind, placebo-controlled trial; July 27, 2020 to October 23, 2020	US (99 sites)	18 years or older persons at high risk for SARS-COV-2 infection, a high risk of severe COVID-19 or both	30,420 (52.7)	mRNA-1273 lipid nanoparticle (LNP)-encapsulated modified RNA encoding the perfusion stabilized full-length spike protein of the SARS-CoV-2 virus): 100 μg (n=15,210)	Saline placebo (n=15,210)	-25° to -15°C 2-8°C for 30 days; room temperature ≤ 12h	Two doses 28 days apart in intramuscular in the deltoid muscle	Efficacy of mRNA-1273 against confirmed SARS-CoV-2 with onset at least 14 days after the second dose in participants who had not previously been infected with SARS-CoV-2 virus (seronegative).	National Institute of Allergy and Infectious Diseases; Moderna	High
Al Kaabi, 2021 (NCT04510207, ChiCRT2000034780) Phase 3 randomized, double-blind; July 16, 2020 to December 31, 2020	United Arab Emirates (UAE), Bahrain, India, China, Syria, Nepal, Egypt, Pakistan, Philippine, and Bangladesh (10 sites)	18 years or older without prior known history of SARS-CoV, SARS-CoV-2, or Middle East respiratory syndrome infection	40,411 (84.4%)	Two inactivated SARS-CoV-2 strains created from Vero cells with aluminum hydroxide adjuvant (WIV04 and HB02) WIV04: 5 μg (n=13,470) HB02: 4 μg (n=13,470)	Aluminum hydroxide (alum)-only (n=13,471)	2-8°C; lifespan unknown	Two doses 21 days apart in intramuscular in the deltoid muscle	Efficacy against laboratory-confirmed symptomatic COVID-19 cases 14 days following a second dose among participants who had no virologic evidence of SARS-CoV-2 infection at randomization	National Key Research and Development Project of China (2020YFC082100); Wuhan Institute of Biological Products Co. Ltd., Beijing Institute of Biological Products Co. Ltd. (Sinopharm).	High
Logunov, 2020 (NCT04530396), Phase 2–3 randomized, double-blind, placebo-controlled trial; September 7, 2020 to November 24, 2020	Russia, Moscow (25 sites)	Participants aged at least 18 years, with negative SARS-CoV-2 PCR and IgG and IgM tests, no infectious diseases in the 14 days before enrollment, and no other vaccinations in the 30 days before enrollment	21,862 (56)	Heterologous prime-boost which combined two vector vaccine based on rAd type 26 (rAd26) and rAd type 5 (rAd5) carrying the gene for SARS-CoV-2 full-length glycoprotein S Prime: rAd26 Boost: rAd5 (n=16,427)	Saline placebo (n=5435)	-18°C (Liquid form); 2-8°C (freeze dried) for up to 6 months	Two doses 21 days apart in intramuscular in the deltoid muscle	Efficacy of vaccine measured as the proportion of participants with COVID-19 confirmed by PCR from day 21 after receiving the first dose.	Moscow City Health Department, Russian Direct Investment Fund, Sberbank, and RUSAL	High
Sadoff, 2021 (NCT04505722) Phase 3 randomized, double-blind, placebo-controlled trial; (ENSEMBLE), September 21, 2020 to November 24, 2020	US, Chile, Peru, Mexico, Argentina, Brazil, Colombia, South Africa	Participants with good or stable healthy, without coexisting conditions aged 18 years or older	43,783 (54.9)	Ad26.COV2.S: replication-incompetent adenovirus type 26 (Ad26) vectored vaccine encoding a stabilized variant of the SARS-CoV-2 S protein (5×1010 VP; n=21,895)	Saline placebo (n=21,888)	-20°C; 2-8°C for 3 months	Single dose in intramuscular in the deltoid muscle	Co-primary efficacies endpoints against the first occurrence of centrally confirmed, moderate to severe/critical COVID-19 with onset at least 14 and 18 days in the per-protocol population who had tested negative for SARS-CoV-2	Janssen/Johnson & Johnson	High
Heath, 2021 (EudraCT number, 2020-004123-16) Phase 3 randomized, observer-blind, placebo-controlled trial (2019nCoV-302); September 28, 2020 to November 28, 2020	UK (33 sites)	Participants aged 18 and 84 years with health or stable chronic medical conditions (HIV, cardiac and respiratory diseases)	15,185 (51.6)	NVX-CoV2373: recombinant nanoparticle encoding the full-length spike glycoprotein of the prototype strain plus Matrix-M adjuvant (5 μg of NVX-CoV2373 plus 50 μg of Matrix-M adjuvant; n=7,593)	Saline placebo (n=7,594)	2-8°C for 3 months	Two doses 21 days apart in intramuscular in the deltoid muscle	Efficacy of virologically confirmed mild, moderate, or severe SARS-CoV-2 infection with an onset at least 7 days after the second dose in participants who were serologically negative at baseline	Novavax	High
Tanriover, 2021 (NCT04582344) Phase 3 randomized, double-blind, placebo-controlled trial; September 14, 2020 to January 05, 2021	Turkey (24 sites)	Volunteers aged 18-59 years without history of COVID-19 and with negative PCR and antibody test results for SARS-CoV-2	10,214 (57.8)	CoronaVac: inactivated whole-virion SARS-CoV-2 vaccine (3 μg of SARS-CoV-2 virion plus 0.45 mg/ml of aluminium hydroxide; n=6,646)	Placebo (n=3,568)	2-8°C for 12-14 h	Two doses 14 days apart in intramuscular in the deltoid muscle	Efficacy of PCR-confirmed symptomatic COVID-19 at least 14 days after the second dose in the per-protocol population	Turkish Health Institutes Association	High

COVID-19, coronavirus disease; US, United States; UK, United Kingdom; HIV, human immunodeficiency virus; HBV, hepatitis B virus; HCV, hepatitis C virus; RT-PCR, reverse transcriptase-polymerase chain reaction; LD, low dose; SD, standard dose; VP, viral particle

Sensitivity analysis, Heterogeneity, and Consistency

After grouping vaccines according to their type, results were similar to those of the main analysis. Except from randomization to 21 days after dose 1, mRNA vaccines were the most effective with an incidence reduction of 57% to 96%, while during this interval, DNA vaccines reduced the COVID-19 incidence by 43% and 46% compared with control and recombinant protein vaccines (Table 2). Because only one single closed loop due to the presence of one direct between-vaccine comparison (HB02 and WIV04) was available in our sample, as shown in Fig. 2, we were unable to compute the global heterogeneity for both primary outcome and secondary outcomes.

Table 2

League table of pairwise comparisons in network meta-analysis for COVID-19 vaccine efficacy from sensitivities analyses

After the first dose
P-score 1.000
mRNA	0.713
0.43 (0.28- 0.65)	Recombinant	0.513
0.34 (0.25- 0.45)	0.79 (0.55- 1.15)	DNA	0.274
0.29 (0.21- 0.39)	0.68 (0.45- 1.00)	0.85 (0.67- 1.08)	Inactivated	0.000
0.13 (0.10- 0.16)	0.30 (0.21- 0.42)	0.37 (0.33- 0.43)	0.44 (0.36- 0.53)	Control
Randomization to 21 days after dose 1
0.908
DNA	0.808
0.97 (0.65-1.43)	mRNA	0.457
0.70 (0.49-1.00)	0.72 (0.45-1.16)	Inactivated	0.167
0.57 (0.48-0.67)	0.59 (0.41-0.83)	0.81 (0.59-1.11)	Control	0.159
0.54 (0.34-0.87)	0.56 (0.32-0.99)	0.78 (0.45-1.34)	0.96 (0.62-1.49)	Recombinant
Starting 7 days after dose 2
0.963
mRNA	0.785
0.62 (0.25- 1.52)	Recombinant	0.460
0.23 (0.13- 0.40)	0.37 (0.16- 0.85)	Inactivated	0.293
0.21 (0.12- 0.36)	0.34 (0.15- 0.77)	0.92 (0.61- 1.41)	DNA	0.000
0.05 (0.03- 0.08)	0.08 (0.04- 0.18)	0.22 (0.16- 0.30)	0.24 (0.18- 0.32)	Control

Vaccines are ordered in the rank of their chance of being the best vaccine. Vaccine estimates are provided as incidence risk ratio (IRR) with 95% confidence interval. Comparisons between vaccines should be read left to right, and their IRR is in the cell in common between the column-defining vaccine and the row-defining vaccine. IRRs < 1 favor the column-defining vaccine for the network estimates. The values above the vaccines are the corresponding P-scores; Significant pairwise comparisons are highlighted (Bolded P values are < 0.05). mRNA: messenger RNA vaccines (BNT162b2 and mRNA-1273); DNA: adenoviral vaccines (Ad26.COV2.S, ChAdOx1 nCoV-19, and Gam-COVID-Vac); Inactivated vaccines (WIV04, HB02, and CoronaVac); and recombinant nanoparticle (NVX-CoV2373).

Discussion

We provide information on the dynamics of vaccine efficacy at different time-points. Our findings provide evidence of higher short-term efficacy of mRNA vaccines, especially the mRNA-1273 vaccine, in reducing the incidence of COVID-19 at any time-point after dose 1. These findings are consistent with the reported VE of 76% (58–87%) for mRNA-1273 (Puranik et al. 2021), 42% (13–62%) (Puranik et al. 2021), and 88% (85–90%) for BNT162b2, 67% (61–72%) for ChAdOx1 nCov-19 (Lopez Bernal et al. 2021), and 75.7% (69.3–80.8%) for the pooled VE from 17 studies (Harder et al. 2021) against symptomatic COVID-19 caused by the delta variant. In addition, at least one week after the second dose, we found a similar protection rate against COVID-19 infection to the reported VE by Sharif of 73% (69–77%) for adenovirus vector vaccine and 85% (82–88%) for the mRNA vaccine (Sharif et al. 2021). Despite this similarity, our study has the advantage of having taken into account the dynamic nature of this vaccine effectiveness, which indicates a rapid increased protection rate after the second dose for mRNA vaccine compared to the DNA and Inactivated vaccines.

Although mRNA vaccines seem to display very similar results, DNA vaccines appear to be more heterogeneous. Taken as a group, DNA vaccines are the most efficient in the first 3 weeks after vaccination, but that is mostly due to the good results of Sputnik-V and ChAdOx1 nCov-19 during this time-period, while Ad26.COV2.S displays the lowest efficacy and performs significantly worse than three other vaccines. This is particularly intriguing given the sponsor strategy of recommending a single injection in the primary vaccination, while all other vaccines offer a 2-injection primary vaccination. Further comparison would require more prolonged data with one-injection regimens for the other vaccines, which is not currently available. Nevertheless, the results of this analysis do not support a one-shot primary vaccination schedule for Ad26.COV2.S.

The strength of this study includes reconstructed IPD to allow vaccines efficacies comparison at different time-points, thereby reducing differences due to the definition of population for analyses and statistical methods, and the accounting for difference in study design and background risk of COVID-19 during the study. The extracted data exactly matches those reported by authors, suggesting the robustness of our results. However, the small number of randomized studies can be a limitation. This lack of sufficient data on mixed comparison between vaccines makes it challenging to assess a possible incoherence between direct and indirect comparisons, which is the statistical manifestation of intransitivity. Nevertheless, transitivity assumption is also addressed by indirectness that refers to the relevance of the included studies to the research question, which was well considered in our study. Therefore, our findings have a great confidence after considering the within-trials bias, reporting bias, indirectness, and imprecision domains of the Confidence in Network Meta-Analysis (CINeMA) approach (Nikolakopoulou et al. 2020). Additionally, our findings should be interpreted with caution because vaccines are compared using the currently available trial interim data with disparate study population, duration of exposure, type of control, definition and assessment of the primary endpoints, and the high trials risk of bias due to per protocol analysis.

The reduction in vaccine effectiveness, combined with the gap between mass vaccination and pandemic progression, raises questions about herd immunity and reinforces vaccine hesitancy. Therefore, the readjustment of vaccination strategies and policies, especially the possibility to administer booster doses of vaccine, and to develop variant-targeted vaccines are urgently needed to overcome this pandemic.

Conclusion

Among the current COVID-19 vaccines, mRNA-1273 provides a higher protection against COVID-19. Adherence to public health guidelines and long-term surveillance of vaccine efficacy and safety are necessary, especially in the context of circulation of variants of concern.

(PDF 539 kb)