Efficacy and safety of COVID-19 vaccines: A network meta-analysis.

OBJECTIVE: Several vaccines showed a good safety profile and significant efficacy against COVID-19. Moreover, in the absence of direct head to head comparison between COVID-19 vaccines, a network meta-analysis that indirectly compares between them is needed.
METHODS: Databases PubMed, CENTRAL, medRxiv, and clinicaltrials.gov were searched. Studies were included if they were placebo-controlled clinical trials and reported the safety profile and/or effectiveness of COVID-19 vaccines. The quality of the included studies was assessed using the Revised Cochrane risk-of-bias tool for randomized trials and the Revised Cochrane risk-of-bias tool for nonrandomized trials.
RESULTS: Forty-nine clinical trials that included 421,173 participants and assessed 28 vaccines were included in this network meta-analysis. The network meta-analysis showed that Pfizer is the most effective in preventing COVID-19 infection whereas the Sputnik Vaccine was the most effective in preventing severe COVID-19 infection. In terms of the local and systemic side, the Sinopharm and V-01 vaccines were the safest.
CONCLUSION: We found that almost all of the vaccines included in this study crossed the threshold of 50% efficacy. However, some of them did not reach the previously mentioned threshold against the B.1.351 variant while the remainder have not yet investigated vaccine efficacy against this variant. Since each vaccine has its own strong and weak points, we strongly advocate continued vaccination efforts in individualized manner that recommend the best vaccine for each group in the community which is abundantly required to save lives and to avert the emergence of future variants.

INTRODUCTION

The Coronavirus disease (COVID‐19) has caused over one hundred million cases and several millions of deaths since it was declared as a pandemic by the World Health Organization (WHO) on March 11, 2020. With a 2%–3% fatality rate, this pandemic resulted in catastrophic effects on health as some countries started to report a decrease in their life expectancy. Since December 2020, several variants with a high number of mutations have been reported in many countries and some of these variants were considered by the WHO as variants of concerns which include Alpha (B.1.1.7), Beta (B.1.351), Gamma (P.1), Delta (B.1.617.2), and most recently, the Omicron (B.1.1.529).

In the absence of effective treatment methods, it was important to develop a vaccine that responded to the emerging pandemic. Several Vaccines were developed using different techniques like live attenuated virus, inactivated virus, viral vectors, recombinant protein, peptide‐based, virus‐like particles, and DNA‐ and RNA‐based vaccines. According to the WHO draft landscape of COVID‐19 candidate vaccines, 184 vaccines are being tested in preclinical trials and 92 in clinical trials and many of these vaccines are being administered all over the world. However, major concerns surround the ability of the vaccines to protect from the variants mentioned above.

Several vaccines showed good safety profile and significant efficacy against COVID‐19 in clinical trials and since regulatory and medical decisions are usually based on benefit risk calculations, defining the appropriate vaccine for different population groups is crucial. Moreover, in the absence of direct head to head comparison between COVID‐19 vaccines and their types, a meta‐analysis that indirectly compares between them through the common component placebo is needed. Several studies have been conducted regarding COVID‐19 vaccines safety and efficacy; however, six of them were merely direct and not comparative (network) meta‐analyses. , , , , , Amongst the other two studies we found, , one was a network meta‐analysis which only studied vaccine efficacy in Phase III clinical trials and did not account for safety or for Phase I and II trials. While the other network meta‐analysis examined vaccine efficacy under generalized nonspecific types such as mRNA vaccines, protein subunit vaccines and viral vector vaccines without separating and comparing each subtype. Hence, we decided to conduct this network meta‐analysis aiming to compare the different COVID‐19 vaccine types and their different subtypes in terms of safety and efficacy and across all trial phases.

METHODS

Registration

In this meta‐analysis, we followed the Preferred Reporting Items for Systematic Reviews and Network Meta‐Analyses (PRISMA‐NMA). This study was prospectively registered in PROSPERO (CRD42021243952).

Search strategy

The search was conducted on March 21, April 15, June 6 and September 2, 2021 and updated on June 27, 2022 by AAT and TNA independently, using the following databases without any language restrictions during our initial search; PubMed, CENTRAL, medRxiv and clinicaltrials.gov. The following keywords were used in the search; ((COVID‐19) OR (2019 Novel Coronavirus Disease) OR (2019 Novel Coronavirus Infection) OR (2019‐nCoV Disease) OR (2019‐nCoV Infection) OR (COVID‐19 Pandemic) OR (COVID‐19 Pandemics) OR (COVID‐19 Virus Disease) OR (COVID‐19 Virus Infection) OR (COVID19) OR (Coronavirus Disease 2019) OR (Coronavirus Disease‐19) OR (SARS Coronavirus 2 Infection) OR (SARS‐CoV‐2 Infection)) AND ((Vaccine) OR (Vaccines)) AND ((Trials) OR (Trial)). The detailed search strategy is described in Supplementary Material 1. Afterward the search results were cross matched and any discrepancy was solved by discussion. The search results, after cross matching, were imported to Zotero reference management software (www.zotero.org) and duplicates were removed.

Selection process

The studies were included if they were placebo‐controlled clinical trials in design and reported the safety profile and/or effectiveness of COVID‐19 vaccines, and only studies published in English were included. Any study that did not meet these criteria was excluded from our analysis. The studies were first screened using title and abstract then the remaining relevant studies were tested against the inclusion criteria using their full‐text form. The study selection was done by AAT and TNA independently and any discrepancy was solved by discussion. The intervention of interest was COVID‐19 vaccines among aged 18 years of age or older and the outcome of interest was safety of the vaccine and its efficacy against COVID‐19 infection. The safety was represented by local, systemic and unsolicited side effects reported by the study participants within 7 days from receiving the first and second doses for two dose vaccines, or the first dose alone for the single dose vaccines. The efficacy of the vaccines were assessed on 3 endpoints. First, the efficacy of the vaccine in preventing COVID‐19 infection which was defined as the ability of the vaccine to prevent COVID‐19 infection detected by regular RT‐PCR testing regardless of the occurrence of symptoms. Second, the efficacy of the vaccine in preventing symptomatic infection which was defined as the occurrence of any COVID‐19‐related symptoms in a participant who had a positive RT‐PCR COVID‐19 test. Third, the efficacy of the vaccine in preventing COVID‐19 severe infection which was defined as the occurrence of any COVID‐19 symptoms with any sign of severe systemic illness including; respiratory failure; evidence of shock; significant acute renal, hepatic, or neurologic dysfunction; admission to an intensive care unit, or death. All the efficacy endpoints were considered if they occurred at least 7 days after the full regimen of the vaccine.

Data extraction and risk of bias assessment

The variables of interest were extracted By AAT and TNA independently then, checked by YYO and SMH and any discrepancy was solved by discussion. The extracted variables were; vaccine name, country, age (median/mean and standard deviation), number of participants, male to female ratio, type of vaccine, type of adjuvant used, comorbidities, phase of the trial, number of placebo and treatment patients who developed the disease, number of placebo and treatment patients who developed symptoms, number of placebo and treatment patients who developed severe disease, number of placebo and treatment patients who died, number of elderly (above 65) placebo and treatment patients who developed the disease, whether neutralization test performed or not, number of placebo and treatment patients who developed local side effects, systemic side effects and unsolicited side effects, the most commonly reported local, systemic and unsolicited side effects and time period. After the data was extracted, the risk of bias of the included randomized trials was assessed using the Revised Cochrane risk‐of‐bias tool for randomized trials (RoB2) while the risk of bias of the included nonrandomized trials was evaluated using Risk Of Bias In Non‐Randomized Studies of Interventions (ROBINS‐I). The risk of bias assessment was also done by AAT and TNA independently then checked by YYO and SMH and any difference in the scoring was solved by discussion.

Data analysis

After data extraction, the relative risk and its corresponding confidence intervals were calculated using the Altman et al. equation for all outcomes and if any zeros had been encountered in the outcomes, 0.5 was added to all cells. In addition, regarding the outcomes related to side effects, if any study had a number of events (side effects) that exceeded the number of the participants in the placebo and treatment groups, the study was excluded from the analysis from that safety outcome as such studies do not provide any information about the relative risk of the event. First, pairwise meta‐analysis was conducted when at least two studies had the same intervention (vaccine) and control (placebo) for a particular outcome. When the network nodes were formed, a network meta‐analysis was carried out to compare different COVID‐19 vaccines using the same component, placebo. The number of nodes was the same as the number of vaccines since each node represented one vaccine. All the COVID‐19 vaccines were compared in the network meta‐analysis and no certain vaccines or treatments were preferred or avoided. Since no studies directly compared between COVID‐19 vaccines, the methods we used to compare the interventions were purely indirect network meta‐analysis methods. The geometry of our network meta‐analysis can be visualized in the Supplementary Material Figures 2–5. The transitivity in the network meta‐analysis was assessed across all the networks by evaluating the similarity of the common comparator (placebo) and the common effect modifiers on the outcomes including; age, gender and comorbidities in the included trials. This was done through evaluation of the included trials by AAT and TNA independently and any disagreement was solved by discussion. Furthermore, we conducted several analyses across our outcomes to compare different types of vaccines. This was done by pooling the trials that explored the same type of vaccine then when network nodes were formed, a network meta‐analysis was performed to compare between different types of vaccines. The treatment nodes were formed by AAT, TNA, YYO, and SMH. The analysis was done by creating six network models incorporating the efficacy outcomes and six network models incorporating the safety outcomes. The efficacy networks evaluated the efficacy of COVID‐19 vaccines and COVID‐19 vaccine types across the following outcomes; COVID‐19 infection, COVID‐19 symptomatic infection, COVID‐19 severe infection. Whereas, the safety networks evaluated the safety of COVID‐19 vaccines and COVID‐19 vaccine types across the following outcomes; local side effects, systemic side effects and unsolicited side effects. Additionally, subanalysis network models were created to compare the COVID‐19 vaccines and their types among elderly across the same aforementioned efficacy and safety outcomes. On top of that, another model was created to compare the efficacy of COVID‐19 vaccines against the B.1.351 variant among those of them which provided such data. The network meta‐analysis was done by applying the Bucher method , using the random effect model. , The inconsistency of the network meta‐analysis was not assessed as all of the ones in our study were from indirect evidence. Additionally, a funnel plot was used to detect publication bias. Meta XL, version 5.3 (EpiGear International, Queensland, Australia) was used in the data analysis. In addition, robvis software was used to create risk of bias plots.

RESULTS

Search results

The search yielded 7245 articles, of which 551 were duplicates. From the remaining 6694 articles, 6241 were excluded by title/abstract screening because they were retracted, case reports and series, letters, in vitro and animal studies, cohort and case‐control studies, cross‐sectional studies, protocols, reviews or irrelevant. 453 articles were reviewed in their full‐text form, from which 402 were excluded because they were excluded because they were combined studies for already included clinical trials, ongoing trials, lack of control group, no efficacy and/or safety data, still recruiting trials, registration for already included trials, trials of booster shots, trials which included participants below 18 years of age, trials studying mixing more than one type of vaccine, terminated trials or trials which studied partial vaccination regimen. The citations of the full list of the excluded studies are provided in the Supplementary Material 2. Finally, 51 articles were included in data extraction, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , 3 of them , , were excluded because in both of them , the number of the events exceeded the number of participants in the treatment and placebo groups and such studies do not provide any information about the relative risk of the event. The other one included HIV patients solely and no other trials included/provided data about HIV patients thus it was excluded. As a result, 49 trials from 48 articles , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , that included 421,173 participants and assessed 28 vaccines of 9 different types were included in the final analysis. The detailed selection process is described in Figure S1.

Summary of network geometry

The network geometry can be visualized in Figures S2–S5. Regarding the network for the comparison of COVID‐19 vaccines, CoronaVac was represented by five trials and Novavax was represented by four trials. Whereas Astrazeneca and Pfizer were represented by three trials each. Furthermore, Sinopharm, Baharat, J&J, Moderna, V‐01, Cansino, Clover, Curevac, Sanofi, ZF2001, and IMBCAMS were examined by two trials for each one of them. Additionally, one trial represented each of the following vaccines: KNCOV, QazCovid‐in, ARCT2001, SCALMP, Sputnik, COH04S1, NDV‐HXP‐S, Spikogen, ZyCov‐D, BIV1‐CovIranand V591 vaccines. Regarding the network for the comparison of COVID‐19 vaccine types, inactivated vaccines were represented by 17 trials while mRNA‐based vaccines and viral vector ones were represented by 8 trials and 10 trials, respectively. Recombinant protein vaccines were evaluated by 8 trials whereas protein subunit vaccines were evaluated by 6 trials. Lastly, virus‐like particle vaccines, DNA, modified Vaccinia Ankara and spike glycoprotein clamp vaccines were represented by one trial each. Moreover, the geometry showed that all the comparisons were indirect which might result in a poorly connected network structure.

Characteristics of the included studies

The characteristics of the included studies are described in Table 1. Twenty‐five trials were conducted in Asia (50%), seven in North and South America (14%), 11 across multiple continents (22%), 5 in Europe (10%), and 2 in Africa (4%). The most documented comorbidities in the included trials were diabetes and chronic respiratory and cardiovascular diseases as 15 studies included participants with diabetes and 14 studies included participants having respiratory diseases and 12 cardiovascular diseases. Coronavac was assessed in five trials, Novavax in four trials, Pfizer and Astrazeneca in three trials for each of them, Moderna, J&J, Baharat, V‐01, Sinopharm, Cansino, Clover, Curevac, Sanofi, ZF2001, and IMBCAMS each of them was assessed in two trials and each of the remaining vaccines was assessed in one trial. The most two types of vaccines that were assessed in the included trials were the inactivated and viral vector vaccines as each one of them was examined by 14 and 10 trials, respectively. Furthermore, the most used adjuvant in the included trials was Alum as 17 (34%) of the included studies used it. In addition, most of the included trials were Phase I trials (42%) and used neutralization tests as one of their outcomes (74%).

TABLE 1

Summary of the characteristics of the included studies

Characteristics
Geographical region
Asia	25 (50)
Europe	5 (10)
North and South America	7 (14)
Africa	2 (4)
Multicontinent	11 (22)
Gender (female: male)
> 2:1	0
2:1	14 (28)
1:1	31 (62)
1:2	3 (6)
1: >2	2 (4)
Comorbidities
Cardiovascular disease	12 (24)
Diabetes	15 (30)
Hypertension	8 (16)
HIV/AIDS	2 (4)
Liver disease	8 (16)
Cerebrovascular disease	4 (8)
Chronic respiratory disease	14 (28)
Cancer	5 (10)
Study design
Nonrandomized controlled trial	2 (4)
Randomized controlled trial	48 (96)
Frequency of interventions examined:
Sinopharm (inactivated virus)	2 (4)
Astrazenica (viral vector)	3 (6)
Baharat Biotech International (inactivated virus (vero cell))	2 (4)
Cansino (viral vector)	2 (4)
Clover Biopharmacuticals COVID‐19 Vaccine (protein subunit)	2 (4)
CoronaVac (inactivated virus)	5 (10)
CureVac (mRNA based)	2 (4)
IMBCAMS (inactivated virus (vero cell))	2 (4)
J&J (viral vector)	2 (4)
Moderna (mRNA based)	2 (4)
NovaVax (recombinant protein)	4 (8)
Pfizer (mRNA based)	3 (6)
Sanofi (recombinant protein)	2 (4)
Sputnik (viral vector)	1 (2)
ZF2001 (recombinant protein)	2 (4)
SCALMP (spike glycoprotein clamp)	1 (2)
Medicago (virus‐like particle)	1 (2)
V‐01 (protein subunit)	2 (4)
ARCT 2001 (mRNA based)	1 (2)
MVC COVID‐19 (protein subunit)	1 (2)
QazCovid‐in (inactivated virus)	1 (2)
KNCOV (inactivated virus)	1 (2)
COH04S1 (modified Vaccinia Ankara)	1 (2)
NDV‐HXP‐S (nonreplicating viral vector)	1 (2)
Spikogen (protein subunit)	1 (2)
ZyCov‐D (DNA)	1 (2)
BIV1‐CovIran (inactivated virus)	1 (2)
V591 (replicating viral vector)	1 (2)
Frequency of vaccines types examined
Inactivated vaccines	14 (28)
mRNA‐based vaccines	8 (16)
Viral vector‐based vaccines	10 (18)
Recombinant protein vaccines	8 (16)
Protein subunit vaccines	6 (12)
Virus‐like particle	1 (2)
Spike glycoprotein clamp	1 (2)
DNA vaccines	1 (2)
Modified Vaccinia Ankara	1 (2)
Outcomes
COVID‐19 infection	9 (18)
COVID‐19 symptomatic infection	15 (30)
COVID‐19 severe infection	17 (34)
COVID‐19 infection among elderly	8 (16)
Local adverse event	49 (98)
Local adverse event among elderly	13 (26)
Systemic side effects	40 (80)
Systemic side effects among elderly	11 (22)
Unsolicited side effects	33 (66)
Neutralization test
No	13 (26)
Yes	37 (74)
Types of adjuvant
CPG	4 (8)
Alum	17 (34)
AS03	3 (6)
Matrix M1	1 (2)
MF‐59	1 (2)
No adjuvant	15 (30)
Trial phases
Phase I	21 (42)
Phase II	10 (20)
Phase III	19 (38)
Recent side effects (thrombosis, capillary leak syndrome and myocarditis) for Astrazeneca, J&J, Pfizer and Moderna:
Thrombosis	Astrazeneca: 4 cases J&J: 11 cases Pfizer: none Moderna: 4 cases
Capillary leak syndrome	Astrazeneca: none J&J: none Pfizer: none Moderna: none
Myocarditis	Astrazeneca: none J&J: none Pfizer: none Moderna: none

Efficacy against COVID‐19 infection

The model that assessed the efficacy of the vaccines against COVID‐19 infection included 216,368 participants from ten trials that examined Moderna, Astrazeneca, Coronavac, Baharat, Sinopharm, Clover, Cansino, Curevac, and ZyCov‐D vaccines only (Table 2). This model showed that the Moderna (RR = 0.12, 95% CI: 0.08–0.17), Sinopharm (RR = 0.31, 95% CI: 0.23–0.42), ZyCov‐D (RR = 0.33, 95%CI: 0.20–0.54), Cansino (RR = 0.36, 95% CI: 0.26–0.47), Baharat (RR = 0.32, 95% CI: 0.19–0.53), and Curevac (RR = 0.54, 95% CI: 0.42–0.70) vaccines were significantly more effective in preventing COVID‐19 infection than placebo, whereas the Coronavac (RR = 0.54, 95%CI: 0.27–1.06) and Astrazeneca (RR = 0.58, 95%CI: 0.32–1.06) vaccines were not significantly more effective than placebo in preventing COVID‐19 infection. The results of the network meta‐analysis are demonstrated in Figure S6.

TABLE 2

The summary of the meta‐analyses of the included studies

		Study group (no. of events/total no.)
Treatment comparison, reference	No. of studies (no. of patients)	Treatment	Control	Risk ratio (95%CI)	The most common reported side effects across the trials
Confirmed COVID‐19 infection: 10 CTs, 216,368 participants
Astrazenica vs. placebo	2 (9297)	75/4628	143/4669	0.58 (0.32–1.06)	–
Moderna vs. placebo	1 (28,207)	26/14,134	224/14,073	0.12 (0.08–0.17)	–
Sinopharm vs. placebo	1 (38,233)	73/25,496	116/12,737	0.31 (0.23–0.42)	–
Coronavac vs. placebo	1 (10,214)	17/6646	17/3568	0.54 (0.27–1.05)	–
Baharat vs. placebo	1 (6289)	19/3248	56/3041	0.32 (0.19–0.53)	–
Clover vs. placebo	1 (30,155)	185/6104	63/6251	0.33 (0.25–0.44)	–
Cansino vs. placebo	1 (21,250)	211/14,586	77/14,591	0.36 (0.28–0.47)	–
Curevac vs. placebo	1 (39,529)	145/12,211	83/12,851	0.54 (0.42–0.71)	–
ZyCov‐D	1 (33,194)	61/13,852	20/13,851	0.33 (0.20–0.54)	–
Consistency H = 1.00
Symptomatic COVID‐19 infection: 15 CTs, 285,528 participants
Astrazenica vs. placebo	3 (13,385)	52/6691	125/6694	0.44 (0.21–0.94)	–
Moderna vs. placebo	1 (28,146)	11/14,134	185/14,073	0.06 (0.03–0.11)	–
Novavax vs. placebo	2 (18,837)	63/9426	183/9411	0.38 (0.23–0.63)	–
Pfizer vs. placebo	1 (40,137)	9/19,965	169/20,172	0.05 (0.03– 0.11)	–
Sputnik vs. placebo	1 (19,866)	16/14,964	62/4902	0.08 (0.05–0.15)	–
Sinopharm vs. placebo	1 (38,233)	47/25,496	95/12,737	0.25 (0.17–0.35)	–
J&J vs. placebo	1 (39,058)	117/ 19,514	351/19,544	0.33 (0.27–0.41)	–
Coronavac vs. placebo	1 (10,214)	9/6646	32/3568	0.15 (0.07–0.32)	–
Baharat vs. placebo	1 (16,973)	24/8471	106/8502	0.23 (0.15–0.35)	–
Clover vs. placebo	1 (30,155)	52/5935	155/5806	0.32 (0.24–0.45)	–
Sanofi vs. placebo	1 (1620)	7/811	18/809	0.39 (0.16–0.92)	–
ZF2001 vs. placebo	1 (28,904)	36/7359	188/7322	0.19 (0.13–0.27)	–
Severe COVID‐19 Infection: 14 CTs, 355,108 participants
Astrazenica vs. placebo	1 (23,745)	0/12,021	1/11,724	0.33 (0.01–7.98)	–
Moderna vs. placebo	1 (28,207)	0/14,134	30/14,073	0.02 (0.00–0.27)	–
Novavax vs. placebo	2 (18,837)	0/9426	5/9411	0.14 (0.02–1.10)	–
Pfizer vs. placebo	1 (40,137)	1/19,965	4/20,172	0.25 (0.03–2.26)	–
Sputnik vs. placebo	1 (19,866)	0/14,964	20/4902	0.01 (0.00–0.13)	–
Sinopharm vs. placebo	1 (38,233)	0/25,496	2/12,737	0.01 (0.00–2.08)	–
J&J vs. placebo	1 (39,058)	14/19,514	60/19,544	0.23 (0.13– 0.42)	–
Coronavac vs. placebo	1 (10,214)	0/6646	1/3568	0.18 (0.00–4.39)	–
Baharat vs. placebo	1 (16,973)	1/8471	16/8502	0.06 (0.01–0.47)	–
Clover vs. placebo	1 (30,155)	0/5935	8/5806	0.06 (0.00–1.00)	–
Cansino vs. placebo	1 (21,250)	1/14,586	25/14,591	0.04 (0.01–0.30)	–
ZF2001 vs. placebo	1 (28,904)	1/7359	13/7322	0.08 (0.01–0.58)	–
Curevac vs. placebo	1 (39,529)	4/12,851	10/12,211	0.38 (0.12–1.21)	–
Consistency H = 1.00					–
COVID‐19 Infection among elderly: 9 CTs, 46,931 participants
Astrazeneca vs. placebo	1 (1006)	1/498	3/508	0.34 (0.04– 3.26)	–
Moderna vs. placebo	1 (7135)	4/3583	29/3552	0.14 (0.05–0.39)	–
Pfizer vs. placebo	1 (15,921)	4/7971	72/7950	0.06 (0.02–0.15)	–
Sputnik vs. placebo	1 (2144)	2/1611	8/533	0.08 (0.02–0.39)	–
Novavax vs. placebo	1 (15,139)	1/1953	9/1957	0.11 (0.01–0.88)	–
Baharat vs. placebo	1 (1858)	5/893	16/965	0.34 (0.12–0.92)	–
Cansino vs. placebo	1 (2670)	10/1323	21/1347	0.48 (0.23–1.03)	–
ZF2001 vs. placebo	1 (417)	7/211	1/206	0.14 (0.02–1.18)	–
Curevac vs. placebo	1 (2499)	12/1319	9/1180	1.19 (0.50–2.82)	–
Consistency H = 1.00
B.1.351 infection symptomatic infection: 2 CTs, 5580 participants
Astrazeneca vs. placebo	1 (1882)	19/944	12/938	0.90 (0.23–1.55)	–
Novavax vs. placebo	1 (3698)	14/1857	24/1841	0.58 (0.30– 1.11)	–
Consistency H = 1.00
Local side effects: 49 CTs, 213,212 participants
Sinopharm vs. placebo	2 (40,731)	6476/27,182	3915/13,533	0.80 (0.77–0.83)	Pain
Astrazenica vs. placebo	1 (2247)	51/1080	21/1167	2.62 (1.59–4.33)	Pain
Baharat vs. placebo	2 (26,147)	724/13,197	662/12,949	1.08 (0.98–1.20)	Pain
Cansino vs. placebo	2 (3665)	1240/1966	317/1699	4.39 (2.01–9.57)	Pain
Clover vs. placebo	2 (1739)	594/923	147/816	7.40 (1.00– 54.68)	Pain
Coronavac vs. placebo	5 (12,412)	865/8320	210/4092	2.06 (1.79–2.37)	Pain
CureVac vs. placebo	2 (4228)	2044/2218	483/2010	3.69 (2.86–4.76)	Pain
IMBCAMS vs. placebo	2 (942)	116/744	7/198	4.20 (1.99–8.88)	Pain
J&J vs. placebo	2 (7541)	405/3998	33/3543	4.13 (2.78–6.14)	Pain
Moderna vs. placebo	2 (29,842)	26,396/15,077	5775/14,765	3.38 (1.88–6.08)	Pain
Novavax vs. placebo	4 (3117)	3054/2750	681/2668	3.75 (2.22–6.34)	Pain
Pfizer vs. placebo	3 (9275)	7512/4720	1049/4555	5.34 (3.46–8.24)	Pain
Sanofi vs. placebo	2 (970)	916/774	202/196	2.47 (0.63 −9.69)	Pain
ZF2001 vs. placebo	2 (29,823)	3484/15,076	2894/14,747	2.13 (0.61 −7.37)	Pain
KNCOV vs. placebo	1 (560)	13/112	61/448	1.17 (0.67 2.06)	Pain
SCALMP vs. placebo	1 (120)	186/96	18/24	1.33 (1.06–1.68)	Pain
Medicago vs. placebo	1 (588)	325/494	14/94	4.42 (2.71–7.19)	Pain
V‐01 vs. placebo	2 (1060)	57/864	27/196	0.47 (0.31–0.74)	Pain
QazCovid‐in vs. placebo	1 (44)	6/22	0/22	13.00 (0.78–217.62	Pain
MVC COVID‐19 vs. placebo	1 (3844)	2381/3295	129/549	3.08 (2.64–3.58)	Pain
ARCT‐2001 vs. placebo	1 (106)	159/78	9/28	3.11 (1.82–5.33)	Pain
COH04S1 vs. placebo	1 (56)	32/51	4/5	0.78 (0.48–1.28)	Pain
NDV‐HXP‐S vs. placebo	1 (210)	162/175	14/35	2.31 (1.54–3.48)	Pain
Spikogen vs. placebo	1 (400)	448/310	38/90	2.37 (1.86–3.02)	Pain
ZyCov‐D vs. placebo	1 (33,194)	18/487	13/447	0.50 (0.19–1.34)	Pain
BIV1‐CovIran vs. placebo	1 (88)	50/76	10/16	1.58 (0.82–3.05)	Pain
V591 vs. placebo	1 (263)	35/210	4/53	2.21 (0.82–5.94)	Pain
Consistency H = 1.00
Local side effects among elderly: 12 CT, 17,115 participants
Clover vs. placebo	1 (60)	37/48	0/12	19.90 (1.31–302.90)	Pain
CoronaVac vs. placebo	1 (37)	13/25	4/12	2.08 (0.82 −5.26)	Pain
J&J vs. placebo	1 (403)	134/322	11/81	3.06 (1.74– 5.39)	Pain
Moderna vs. Placebo	2 (7639)	6194/3892	1074/3747	4.42 (2.51–7.79)	Pain
Novavax vs. placebo	1 (907)	351/461	40/446	8.49 (6.29 −11.46)	Pain
Pfizer vs. placebo	2 (3624)	2972 /1851	368/1773	11.26 (3.05–41.51)	Pain
Sanofi vs. Placebo	1 (139)	168/118	2/21	10.5 (2.81–39.24)	Pain
Medicago vs. placebo	1 (282)	118/235	5/47	4.72 (2.04–10.92)	Pain
V‐01 vs. placebo	1 (180)	0/18	1/72	0.78 (0.03–18.41)	Pain
MVC COVID‐19	1 (3844)	505/720	19/118	4.36 (2.88– 6.59)	Pain
Consistency H = 1.00
Systemic side effects: 40 CT, 168,433 participants
Sinopharm vs. placebo	2 (40,715)	7517/27,182	3748/13,533	1.00 (0.97–1.04)	1 trial: headache 1 trial: fever
Astrazenica vs. placebo	1 (2247)	327/1080	343/1167	1.03 (0.90– 1.17)	Headache
Baharat vs. placebo	2 (26,147)	599/13,197	459/12,949	1.24 (1.10–1.40)	Headache
Cansino vs. placebo	2 (3665)	1629/1954	817/1698	1.38 (1.31 −1.47)	Headache
Clover vs. placebo	2 (1739)	608/923	157/816	2.10 (1.49–2/97)	Headache
CoronaVac vs. placebo	4 (11,978)	1300/8050	617/3928	1.05 (0.96–1.14)	1 Trial: headache 3 Trials: fatigue
IMBCAMS vs. placebo	2 (942)	82/744	21/198	1.01 (0.64–1.59)	Fatigue
J&J Safety	2 (7541)	513/3998	83/3543	2.60 (2.09–3.24)	1 Trial: geadache 1 Trial: fatigue
Moderna vs. placebo	1 (29,842)	20,432/15,077	11,844/14,765	1.41 (1.08–1.85)	Fatigue
Novavax vs. placebo	4 (5417)	2429/2752	1450/2665	1.59 (1.21–2.10)	1 Trial: fatigue 3 Trials: headache
Pfizer vs. placebo	2 (339)	296/252	30/87	2.87 (0.77–10.69)	1 Trial: fatigue 1 Trial: fever
Sanofi vs. placebo	2 (970)	1114/774	93/196	1.56 (1.29–1.88)	Headache
ZF2001 vs. placebo	2 (29,823)	7305/15,076	6905/14,747	1.05 (1.03–1.08)	Fever
KNCOV vs. placebo	1 (560)	36/448	7/112	1.29 (0.59−2.81)	Headache
Medicago vs. placebo	1 (588)	273/494	45/94	1.15 (0.92–1.45)	Fatigue
V‐01 vs. placebo	2 (1060)	146/864	32/196	0.84 (0.60–1.18)	Fatigue
QazCovid‐in vs. placebo	1 (44)	0/22	1/22	3.00 (0.13–69.87)	Fever
MVC COVID‐19	1 (3844)	2559/3295	291/549	1.47 (1.35–1.59)	Fatigue
Curevac vs. placebo	1 (3981)	1881/2003	1255/1978	1.48 (1.43–1.53)	Headache
NDV‐HXP‐S	1 (210)	11/35	131/175	2.38 (1.45–3.91)	Headache
Spikogen vs. placebo	1 (400)	400/310	78/90	1.15 (1.06–1.25)	Headache
ZyCov‐D	1 (934)	6/487	11/447	0.50 (0.19–1.34)	Headache
BIV1‐CovIran vs. placebo	1 (88)	45/76	6/16	1.58 (0.82–3.05)	Headache
V591 vs. placebo	1 (263)	97/210	35/53	0.70 (0.55–0.89)	Headache
Consistency H = 1.00
Systemic side effects among elderly: 11 CT, 13,615 participants
Clover vs. placebo	1 (60)	37/48	4/12	2.31 (1.02– 5.22)	Headache
J&J Safety	1 (403)	162/322	19/81	2.14 (1.43–3.23)	Fatigue
Moderna vs. placebo	2 (7639)	4685/3892	2534/3747	1.48 (1.45–1.51)	Fatigue
Novavax vs. placebo	1 (906)	253/463	107/443	2.26 (1.88 −2.72)	Fatigue
Pfizer vs. placebo	2 (162)	115/120	13/42	2.50 (0.81 −7.72)	1 Trial: fatigue 1 Trial: fever
Sanofi vs. placebo	1 (139)	255/118	10/21	2.10 (1.34– 3.29)	Headache
Medicago vs. placebo	1 (282)	106/235	21/47	1.01 (0.71–1.43)	Fatigue
V‐01 vs. placebo	1 (180)	0/18	7/72	0.26 (0.02–4.29)	Fatigue
MVC COVID‐19	1 (3844)	394/720	42/118	1.54 (1.20–1.98)	Fatigue
Consistency H = 1.00
Unsolicited side effects: 33 CT, 223,575 participants
Sinopharm vs. placebo	2 (40,715)	2943/27,182	145/13,533	1.03 (0.97–1.09)	2 Trials: not stated
Baharat vs. placebo	2 (26,147)	1987/13,197	1726/12,949	1.13 (1.07–1.20)	2 Trial: not stated
Cansino vs. placebo	2 (3667)	366/1967	316/1700	1.09 (0.96–1.25)	Not stated
CoronaVac vs. placebo	3 (11,557)	407/7702	308/3855	0.70 (0.60–0.81)	1 Trial: gastrointestinal symptoms 2 Trials: not stated
CureVac vs. placebo	2 (4241)	1010/2007	898/1987	1.11 (1.04–1.19)	Not stated
IMBCAMS vs. placebo	2 (942)	31/744	3/198	2.83 (0.87 −9.18)	Not stated
J&J Safety	2 (7541)	595/3998	434/3543	1.20 (0.92 −1.56)	Not stated
Moderna vs. placebo	2 (30,751)	3745/15,585	3328/15,166	1.11 (1.07 −1.16)	1 Trial: not stated 1 Trial: nervous system symptoms
Novavax vs. placebo	4 (17,519)	2153/9195	1607/8324	1.23 (1.16–1.30)	Not stated
Pfizer vs. placebo	3 (43,592)	5863/21,874	2651/21,718	2.17 (1.47 −3.23)	Not stated
Sanofi vs. placebo	2 (980)	707/774	137/196	1.54 (0.84–2.81)	Not stated
ZF2001 vs. placebo	1 (950)	32/640	27/310	0.57 (0.35–0.94)	Not stated
KNCOV vs. placebo	1 (560)	10/448	4/112	0.63 (0.20–1.96)	Not stated
SCALMP vs. placebo	1 (120)	35/96	7/24	1.25 (0.64–2.46)	Not stated
V‐01 vs. placebo	1 (180)	216/144	18/36	2.00 (1.44–2.77)	Not stated
MVC COVID‐19	1 (3844)	932/3295	149/549	1.04 (0.90–1.21)	Not stated
Clover vs. placebo	1 (30,128)	2553/15,064	234/15,064	1.09 (1.03–1.15)	Not stated
ZyCov‐D vs. placebo	1 (1418)	121/709	94/709	1.29 (1.00–1.65)	Not stated
Consistency H = 1.00

–

Efficacy against COVID‐19 symptomatic infection

Fifteen clinical trials that included 285,528 participants, reported the efficacy of Pfizer, Moderna, Sputnik, Sinopharm, J&J, Astrazeneca, Coronavac, Baharat, Novavax, Clover, Sanofi, and ZF2001COVID‐19 vaccines against symptomatic infection (Table 2). Pfizer (RR = 0.05, 95%CI: 0.03–0.1), Moderna (RR = 0.06, 95%CI: 0.03–0.11), Sputnik (RR = 0.08, 95%CI: 0.05–0.14), Coronavac (RR = 0.15, 95%CI: 0.07–0.32), ZF2001 (RR = 0.19, 95% CI: 0.07–0.32), Baharat (RR = 0.15, 95%CI: 0.07–0.32), Sinopharm (RR = 0.25, 95%CI: 0.17–0.36), Clover (RR = 0.32, 95% CI: 0.23–0.44), J&J (RR = 0.33, 95%CI: 0.27–0.41), Novavax (RR = 0.38, 95%CI: 0.23–0.63), Sinovac (RR = 0.39, 95% CI: 0.16–0.94), and Astrazeneca (RR = 0.44, 95%CI: 0.21–0.94) had showed statistically significant protective effect against COVID‐19 symptomatic infection. Figure S7 shows the results of the network meta‐analysis.

Efficacy against COVID‐19 severe infection

Fourteen clinical trials that included 355,108 participants, reported the efficacy of Pfizer, Moderna, Sputnik, Astrazeneca, Novavax, Coronavac, Sinopharm, Bahart, Clover, Cansino, ZF2001, and Curevac vaccines against COVID‐19 severe infection (Table 2). Sputnik (RR = 0.01, 95%CI: 0.00–0.11), Sinopharm (RR = 0.01, 95%CI: 0.00–0.20), Moderna (RR = 0.02, 95%CI: 0.00–0.33), Cansino (RR = 0.04, 95% CI: 0.01–0.22), Baharat (RR = 0.06, 95%CI: 0.01–0.41), ZF2001 (RR = 0.08, 95% CI: 0.01–0.61), and J&J (RR = 0.23, 95%CI: 0.13–0.41) vaccines showed statistically significant efficacy against COVID‐19 severe infection. In contrast, Clover (RR = 0.06, 95% CI: 0.00–1.90), Novavax (RR = 0.14, 95%CI: 0.02–1.10), Coronavac (RR = 0.18, 95%CI: 0.01–3.77), Pfizer (RR = 0.25, 95%CI: 0.03–2.17), Astrazeneca (RR = 0.34, 95%CI: 0.04–3.07), and Curevac (RR = 0.38, 95% CI: 0.12–1.21) were not statistically significant in preventing COVID‐19 severe infection. The network meta‐analysis results are shown in Figure S8.

Vaccine types efficacy

The model that compared COVID‐19 vaccine types included 133,720 participants from seven trials (Table 3). Inactivated vaccines (RR = 0.34, 95%CI: 0.26–0.44) and Viral vector (RR = 0.48, 95%CI: 0.32–0.74) showed significant protective effect against COVID‐19 infection compared to placebo, whereas mRNA‐based vaccines did not reach statistical significance levels (RR = 0.26, 95%CI: 0.06–1.12) (Figure S9). Moreover, the model that compared COVID‐19 vaccine types in terms of efficacy against COVID‐19 symptomatic infection was composed of 13 clinical trials with 255,065 participants (Table 3). All mRNA‐based (RR = 0.05, 95%CI: 0.03–0.09), inactivated (RR = 0.24, 95%CI: 0.19–0.31), viral vector (RR = 0.30, 95%CI: 0.16–0.56), and protein subunit (RR = 0.30, 95%CI: 0.21–0.43) vaccines showed significant protection against COVID‐19 infection (Figure S10). Furthermore, in the model that assessed for the effectiveness against COVID‐19 severe infection, which included 309,674 participants from 12 clinical trials (Table 3), all inactivated (RR = 0.05, 95%CI: 0.01–0.21), viral vector (RR = 0.08, 95%CI: 0.02–0.37), protein subunit (RR = 0.10, 95%CI: 0.03–0.37), and mRNA (RR = 0.18, 95%CI: 0.04–0.83) based COVID‐19 vaccines showed significant protective effect. (Figure S11). The results of network meta‐analysis are shown in Figures S9–S11.

TABLE 3

The summary of the meta‐analyses of the COVID‐19 vaccines types

		Study group (no. of events/total no.)		Risk ratio (95%CI)
Treatment comparison, reference	No. of studies (no. of patients)	Treatment	Control
Comparison between COVID‐19 vaccines types against infection: 7 CTs, 133,720 participants
Viral vector vs. placebo	3 (38,474)	152/19,219	354/19,255	0.48 (0.32–0.74)
mRNA vs. placebo	1 (53,269)	109/26,985	369/26,284	0.12 (0.08–0.17)
Inactivated vs. placebo	3 (41,977)	109/22,631	189/19,346	0.34 (0.26–0.44)
Consistency H = 1.00
Comparison between COVID‐19 vaccines types against symptomatic infection: 13 CTs, 255,065 participants
Viral vector vs. placebo	5 (72,309)	185/41,169	538/31,137	0.30 (0.16–0.56)
mRNA vs. placebo	2 (68,344)	20/34,099	354/34,245	0.05 (0.03–0.09)
Inactivated vs. placebo	3 (65,420)	80/40,613	233/24,807	0.24 (0.19–0.31)
Protein subunit vs. placebo	3 (48,992)	115/15,361	338/15,217	0.30 (0.21–0.43)
Consistency H = 1.00
Comparison between COVID‐19 vaccines types against severe infection: 12 CTs, 309,674 participants
Viral vector vs. placebo	4 (111,846)	15/61,090	106/50,770	0.08 (0.02–0.37)
mRNA vs. placebo	3 (83,416)	5/36,950	44/46,466	0.18 (0.04–0.83)
Inactivated vs. placebo	3 (65,420)	1/40,613	19/24,807	0.05 (0.01–0.21)
Protein subunit vs. placebo	2 (48,992)	0/15,358	13/15,217	0.10 (0.04–0.83)
Comparison between COVID‐19 vaccines types infection among elderly: 8 CTs, 37,417 participants
Viral vector vs. placebo	4 (9112)	975/5151	346/3691	0.12 (0.04–0.35)
mRNA vs. placebo	3 (25,694)	20/12,873	110/12,821	0.22 (0.04–1.35)
Inactivated vs. placebo	1 (1858)	5/893	16/965	0.34 (0.12–0.94)
Protein subunit vs. placebo	1 (3910)	1/1953	9/1957	0.13 (0.03–0.57)
Consistency H = 1.00
COVID‐19 vaccines comparison in local side effects: 35 CTs, 162,876 participants
Viral vector vs. placebo	4 (9998)	618/5253	68/4745	3.51 (2.39–5.16)
mRNA vs. placebo	9 (43,959)	36,380/22,171	7,329/21,484	4.07 (3.22–5.13)
Inactivated vs. placebo	12 (65,611)	7992/49,147	4739/16,464	1.55 (1.17–2.05)
Recombinant protein vs. placebo	3 (35,707)	7474/18,636	1095/17,611	2.92 (1.80–4.71)
Protein subunit vs. placebo	7 (7601)	4396/5862	372/1739	1.44 (0.84–2.46)
Consistency H = 1.00
COVID‐19 vaccines comparison in local side effects among elderly: 11 CTs, 13,737 participants
mRNA vs. placebo	4 (11,263)	9166/5743	1442/5520	5.13 (3.76–6.99)
Recombinant protein vs. placebo	2 (1046)	519/579	42/467	8.58 (6.40–11.49)
Protein subunit vs. placebo	3 (988)	542/786	20/202	3.41 (2.21–5.27)
Inactivated vs. placebo	1 (37)	13/25	4/12	2.08 (0.82–5.27)
Viral vector vs. placebo	1 (403)	134/322	11/81	3.06 (1.74–5.39)
Consistency H = 1.00
COVID‐19 vaccines comparison in systemic side effects: 34 CTs, 163,518 participants
Viral Vector vs. placebo	5 (13,213)	1879/6868	1159/6335	1.89 (0.93–3.86)
mRNA vs. placebo	6 (34,670)	23,234/17,714	13,217/16,956	1.46 (1.31–1.63)
Inactivated vs. placebo	10 (79,719)	9543/49,073	4897/30,646	1.07 (0.99–1.16)
Recombinant protein vs. placebo	7 (28,873)	10,848/18,611	8454/17,607	1.44 (1.15–1.80)
Protein subunit vs. placebo	6 (7043)	3455/5392	648/1626	1.40 (1.09–1.79)
Consistency H = 1.00
COVID‐19 vaccines comparison in systemic side effects among elderly: 11 CTs, 10,237 participants
mRNA vs. placebo	4 (7801)	4800/4012	2547/3789	1.48 (1.37–1.60)
Recombinant protein vs. placebo	2 (1045)	508/581	117/464	2.24 (1.89–2.65)
Viral vector vs. placebo	1 (403)	162/322	19/81	2.14 (1.42–3.22)
Protein subunit vs. placebo	4 (988)	431/786	53/202	1.07 (0.26–4.37)
Consistency H = 1.00
COVID‐19 vaccines comparison in unsolicited side effects: 30 CTs, 173,330 participants
mRNA vs. placebo	8 (59,545)	10,787/40,063	6902/19,482	1.37 (1.01–1.84)
Inactivated vs. placebo	8 (69,753)	3700/42,861	810/26,892	0.98 (0.82–1.17)
Recombinant protein vs. placebo	7 (6601)	2564/4185	617/2416	1.23 (1.05–1.44)
Viral vector vs. placebo	3 (3159)	935/5583	743/5117	1.19 (0.89–1.59)
Protein subunit vs. placebo	4 (34,272)	3736/18,599	2487/15,673	1.22 (1.00–1.50)
Consistency H = 1.00

Local side effects

Forty‐nine clinical trials that included 213,212 participants reported the local side effects of the COVID‐19 vaccines (Table 2). Only V‐01 (RR = 0.47, 95%CI: 0.31–0.74), COH04S1 (RR = 0.78; 95%CI: 0.48–1.27), Sinopharm (RR = 0.80, 95%CI: 0.77–0.83), BIVI‐CovIran (RR = 1.05, 95%CI: 0.70–1.58), Baharat (RR = 1.08, 95%CI: 0.98–1.20), KNCOV (RR = 1.17, 95%CI: 0.66–2.08), ZyCov‐D (RR = 1.27, 95%CI: 0.63–2.56), ZF2001 (RR = 2.13, 95%CI: 0.61–7.37), V591 (RR = 2.21, 95%CI: 0.82–5.95), Sanofi (RR = 2.47, 95%CI: 0.63–9.69), and QAZ Covid‐in (RR = 13.00 95%CI: 0.78–217.14) COVID‐19 vaccines were not significantly more harmful in causing local side effects than placebo. On other hand, SCALMP (RR = 1.33, 95%CI: 1.06–1.67), CoronaVac (RR = 2.06, 95%CI: 1.79–2.37), NDV‐HXP‐S (RR = 2.31, 95%CI: 1.54–3.47), Spikogen (RR = 2.37, 95%CI: 1.86–3.02), Astrazeneca (RR = 2.62, 95%CI: 1.59–4.32), MVC COVID‐19 (RR = 3.08, 95%CI: 2.64–3.59), ARCT 2001 (RR = 3.11, 95%CI: 1.82–5.32), Moderna (RR = 3.38, 95%CI: 1.88–6.08), Curevac (RR = 3.69, 95%CI: 2.86–4.76), Novavax (RR = 3.75, 95%CI: 2.22–6.34), J&J (RR = 4.13, 95%CI: 2.78–6.14), IMBCAMS (RR = 4.20, 95%CI:1.99–8.88), Cansino (RR = 4.39, 95%CI: 2.01–9.57), Medicago (RR = 4.42, 95%CI: 2.71–7.20), Pfizer (RR = 5.34, 95%CI: 3.46–8.24), and Clover (RR = 7.40, 95%CI: 1.00–54.68) COVID‐19 vaccines were significantly more harmful than placebo in causing local side effects. The network meta‐analysis results are shown in Figure S12. It is important to mention that all of the included trials reported that pain was the most common local side effect (Table 2).

Systemic side effects

Systemic side effects of COVID‐19 vaccines were documented in 40 clinical trials that included 168,433 participants (Table 2). Of them ZyCov‐D (RR = 0.50, 95%CI: 0.19–1.33), V591 (RR = 0.70 95%CI: 0.55–0.89), V‐01 (RR = 0.84, 95%CI: 0.60–1.18), Sinopharm (RR = 1.00, 95%CI: 0.97–1.04), IMBCAMS (RR = 1.01, 95%CI: 0.64–1.59), Astrazeneca (RR = 1.03, 95%CI: 0.90–1.17), CoronaVac (RR = 1.05, 95%CI: 0.96–1.14), Medicago (RR = 1.15, 95%CI: 0.92–1.44), KNCOV (RR = 1.29, 95%CI: 0.59–2.82), BIVI‐CovIran (RR = 1.58, 95%CI: 0.82–3.05), Pfizer (RR = 2.87, 95%CI: 0.77–10.69), and QazCovid‐in (RR = 3.00, 95%CI: 0.13–69.55) were insignificantly associated with systemic side effects compared to placebo. Differently, ZF2001 (RR = 1.05, 95%CI: 1.03–1.08), NVD‐HXP‐S (RR = 1.15, 95%CI: 1.06–1.25), Spikogen (RR = 1.15, 95%CI: 1.06–1.25), Baharat (RR = 1.24, 95%CI: 1.10–1.40), Cansino (RR = 1.38, 95%CI: 1.31–1.47), Moderna (RR = 1.41, 95%CI: 1.08–1.85), MVC COVID‐19 (RR = 1.47, 95%CI: 1.35–1.60), Sanofi (RR = 1.56, 95%CI: 1.29–1.88), Novavax (RR = 1.59, 95%CI: 1.21–2.10), Clover (RR = 2.10, 95%CI: 1.49–2.97), and J&J (RR = 2.60, 95%CI: 2.09–3.24) COVID‐19 vaccines were significantly more associated with systemic side effects compared to placebo. Figure S13 demonstrates the results of the network meta‐analysis. In addition, it is worth mentioning that the most common reported systemic side effect in these trials was headache as twenty two trials (55%) observed it as the most common side effect. Nevertheless, only 5 and 13 trials reported that fever and fatigue were the most commonly observed systemic side effects, respectively (Table 2).

Unsolicited side effects

Unsolicited side effects were reported in 33 clinical trials that included 223,575 participants (Table 2). From them ZF2001 (RR = 0.57, 95%CI: 0.35–0.93), KNCOV (RR = 0.63, 95%CI: 0.20–1.97), Coronavac (RR = 0.70, 95%CI: 0.60–0.81), Sinopharm (RR = 1.03, 95%CI: 0.97–1.09), MVC COVID‐19 (RR = 1.04, 95%CI: 0.90–1.21), Cansino (RR = 1.09, 95%CI: 0.96–1.25), J&J (RR = 1.20, 95%CI: 0.92–1.56), SCALMP (RR = 1.25, 95%CI:0.64–2.45), Sanofi (RR = 1.54, 95%CI: 0.84–2.81), and IMBCAMS (RR = 2.83, 95%CI: 0.87–9.18) COVID‐19 vaccines were insignificantly associated in producing unsolicited side effects compared to placebo. In comparison, Moderna (RR = 1.11, 95%CI: 1.07–1.16), Curevac (RR = 1.11, 95%CI: 1.04–1.20), Baharat (RR = 1.13, 95%CI: 1.07–1.20), Novavax (RR = 1.23, 95%CI: 1.16–1.30), NVD‐HXP‐S (RR = 1.29, 95%CI: 1.00–1.66), V‐01 (RR = 2.00, 95%CI: 1.44–2.77), and Pfizer (RR = 2.17, 95%CI: 1.47–3.23) vaccines showed a significant association in producing unsolicited side effects. The network meta‐analysis results are demonstrated in Figure S14. Additionally, most of the studies (94%) did not report what was the most commonly observed unsolicited side effect. On the other hand, one trial for Moderna vaccine and another one for Coronavac vaccine stated that the nervous system‐related symptoms and gastrointestinal symptoms were the most observed unsolicited side effects, respectively (Table 2). Furthermore, we looked to the included trials for the recently observed side effects of COVID‐19 vaccines including myocarditis for Moderna and Pfizer as well as thrombosis and capillary leak syndrome for J&J and Astrazeneca vaccines. However, no myocarditis or capillary leak syndrome cases happened in the trials of all the four mentioned vaccines. Differently, 19 cases of thrombosis were observed; 11 cases in J&J vaccine trials, four cases in Astrazeneca and Moderna vaccines trials, while no cases in Pfizer vaccine trials.

Vaccine types safety

Thirty‐five clinical trials that included 162,876 participants were pooled in the model that investigated COVID‐19 vaccines local side effects. Only protein subunit vaccines (RR = 1.44, 95%CI: 0.84–2.46) were insignificantly associated with local side effects compared to placebo. Whereas all of inactivated (RR = 1.55, 95%CI: 1.17–2.05), recombinant protein (RR = 2.92, 95%CI: 1.80–4.71), viral vector (RR = 3.51, 95%CI: 2.39–5.16), and mRNA (RR = 4.07, 95%CI: 3.22–5.13) based vaccines were significantly more harmful than placebo in producing local side effects (Figure S15). Moreover, 163,518 participants from 34 clinical trials were included in the model that assessed for systemic side effects. Inactivated (RR = 1.07, 95%CI: 0.99–1.16) and viral vector (RR = 1.89, 95%CI: 0.93–3.86) vaccines were not significantly associated with systemic side effects compared to placebo. In contrast, protein subunit (RR = 1.40, 95%CI: 1.09–1.79), recombinant protein (RR = 1.44, 95%CI: 1.15 −1.80), and mRNA (RR = 1.46, 95%CI: 1.31–1.63) vaccines were significantly associated with systemic side effects compared to placebo (Figure S16). Unsolicited side effects were investigated using a model that included 173,330 participants from 30 clinical trials. This model showed that inactivated (RR = 0.98, 95%CI: 0.82–1.17) and viral vector (RR = 1.19, 95%CI: 0.89–1.59) vaccines were insignificantly associated with unsolicited side effects while protein subunit (RR = 1.22, 95%CI: 1.00–1.50), recombinant protein (RR = 1.23, 95%CI: 1.05–1.44) and mRNA‐based vaccines (RR = 1.37, 95%CI: 1.01–1.84) were significantly associated with them (Figure S17). The network meta‐analysis results are shown in Figures S15–S17.

Subanalysis

In the subanalysis that investigated the efficacy of Pfizer, Sputnik, Moderna, Novavax, Baharat, Astrazeneca, Cansino, ZF2001, and Curevac COVID‐19 vaccines against infection among elderly, 46,931 participants from 9 clinical trials were pooled (Table 2). Pfizer (RR = 0.05, 95%CI: 0.02–0.13), Sputnik (RR = 0.08, 95%CI: 0.02–0.35), Moderna (RR = 0.14, 95%CI: 0.05–0.39), and Baharat (RR = 0.34, 95%CI: 0.12–0.94) vaccines showed a statistically significant protective effect against COVID‐19 infection, while Novavax (RR = 0.11, 95%CI: 0.01–1.03), ZF2001 (RR = 0.14, 95%CI: 0.02–1.08), Astrazeneca (RR = 0.34, 95%CI: 0.04–3.07), Cansino (RR = 0.48, 95%CI: 0.23–1.02), and Curevac (RR = 1.19, 95%CI: 0.50–2.83) vaccines were not statistically significant in protecting from COVID‐19 infection (Figure S18). Furthermore, the model that compared the efficacy of COVID‐19 vaccine types among elderly in preventing infection included 37,417 participants from eight clinical trials. This model showed that viral vector (RR = 0.12, 95%CI: 0.04–0.35), protein subunit vaccines (RR = 0.13, 95%CI: 0.03–0.57), and inactivated (RR = 0.34, 95%CI: 0.12–0.94) vaccines were significantly effective in preventing infection among elderly. Differently, mRNA‐based vaccines (RR = 0.22, 95%CI: 0.04–1.35) were not significantly effective in preventing infection among elderly (Figure S19). Figures S18 and S19 demonstrate the results of the network meta‐analysis.

In another model that studied the efficacy of Astrazeneca and Novavax COVID‐19 vaccines against the B.1.351 variant inducing symptomatic infection, two clinical trials that included 5580 participants were pooled (Table 2). This model showed that both Novavax (RR = 0.58, 95%CI: 0.30–1.12) and Astrazeneca (RR = 0.90, 95%CI: 0.35–2.34) vaccines were statistically insignificant in preventing symptomatic B.1.351 variant infection. Figure S20 shows the results of the network meta‐analysis.

Twelve trials that included 17,115 participants were pooled in the model that evaluated local side effects of COVID‐19 vaccines among elderly (Table 2). Only V‐01 (RR = 0.78, 95%CI: 0.03–19.23), Clover (RR = 1.31, 95%CI: 0.34–5.11), and CoronaVac (RR = 2.08, 95%CI: 0.82–5.27) vaccines were insignificantly associated in producing local side effects among elderly. On the other hand, J&J (RR = 3.06, 95%CI: 1.74–5.39), MVC COVID‐19 (RR = 4.36, 95%CI: 2.88–6.60), Moderna (RR = 4.42, 95%CI: 2.51–7.79), Medicago (RR = 4.72, 95%CI: 2.04–10.92), Novavax (RR = 8.49, 95%CI: 6.29–11.46), Sanofi (RR = 10.50, 95%CI: 2.81–39.24), and Pfizer (RR = 11.26, 95%CI: 3.05–41.51) vaccines were significantly associated with local side effects among elderly (Figure S21). Furthermore, 13,615 participants from 11 trials were pooled in the model that evaluated systemic side effects of COVID‐19 vaccines among elderly (Table 2). V‐01 (RR = 0.26, 95%CI: 0.02–3.81), Medicago (RR = 1.01, 95%CI: 0.71–1.43), and Pfizer (RR = 2.50, 95%CI: 0.81–7.72) were insignificantly associated with systemic side effects among elderly. While all of Moderna (RR = 1.48, 95%CI: 1.45–1.51), MVC COVID‐19 (RR = 1.54, 95%CI: 1.20–1.98), Sanofi (RR = 2.10, 95%CI: 1.34–3.29), J&J (RR = 2.14, 95%CI: 1.42–3.22), Novavax (RR = 2.26, 95%CI: 1.88–2.72), and Clover (RR = 2.31, 95%CI: 1.02–5.23) were significantly associated with systemic side effects among elderly (Figure S22). The model that compared the local side effects of COVID‐19 vaccine types among elderly included 13,737 participants from 11 clinical trials (Table 3). Only Inactivated vaccines were not significantly associated with local side effects among elderly. In comparison, viral vector (RR = 3.06, 95%CI: 1.74–5.39), protein subunit (RR = 3.41, 95%CI: 2.21–5.27), mRNA (RR = 5.13, 95%CI: 3.76–6.99), and recombinant protein (RR = 8.58, 95%CI: 6.40–11.49) vaccines were significantly more harmful than placebo in producing local side effects among elderly (Figure S23). Results from the model that compared for systemic side effects among elderly and included 10,237 participants from 11 clinical trials, showed that only protein subunit vaccines were not significantly associated with these side effects (RR = 1.07, 95%CI: 0.26–4.37). On the other hand, all of mRNA (RR = 1.48, 95%CI: 1.37–1.60), viral vector (RR = 2.14, 95%CI: 1.42–3.22) and recombinant protein (RR = 2.24, 95%CI: 1.89–2.65) vaccines were significantly more harmful than placebo (Figure S24). The results of the network meta‐analysis are demonstrated in Figures S21–S24.

Publication bias and risk of bias assessment

Publication bias funnel plots revealed asymmetry in both COVID‐19 vaccines safety and efficacy (Figures S25 and S26). The summary of the distribution of Cochrane collaboration risk of bias assessment showed that only three of the included studies had some concerns about risk of bias due to deviation from the intended intervention and one clinical trial had some concerns due to bias in the measurement of the outcome. In addition, one clinical trial showed some concerns and another six trials showed high risk of bias arising from the randomization process. Forty trials (80%) showed low overall risk of bias whereas six trials showed high overall risk of bias and four trials showed some concerns overall risk of bias (Figure S27). The detailed risk of bias assessment for each study is described in Figure S28.

DISCUSSION

This network meta‐analysis studied several controlled clinical trials investigating the efficacy and safety of various COVID‐19 vaccines and was able to yield results concerning different aspects of these vaccines. To start with, we reported that the most effective vaccine against COVID‐19 infection is Moderna at an efficacy of 88% followed by Sinopharm at 69% and Baharat at 68% while the least effective were Coronavac at 6%, Curevac at 46% and Astrazeneca at 42% as data against COVID‐19 infection was only reported by these five vaccines. In terms of vaccine efficacy against symptomatic COVID‐19 infection, the most effective vaccine was Pfizer at 95%, closely followed by Moderna at 94%. On the other hand, Astrazeneca proved to be the least effective at 56%, this low efficacy can be explained by the fact that this efficacy of Astrazeneca vaccine included the efficacy against B.1.351 variant In addition to this, after reviewing several cohort studies related to Pfizer vaccine, it was revealed that their vaccine efficacy against symptomatic COVID‐19 infection was 94%, which is similar to our findings of 95% efficacy. Moreover, we studied the efficacy of vaccines against severe COVID‐19 infection and found that Sputnik was the most effective one at 99% with Sinopharm and Moderna closely behind at 99% and 98% respectively. Meanwhile Astrazeneca and Curevac were the least effective at 66% and 62%, respectively. In the comparison between COVID‐19 vaccine types, mRNA‐based vaccines were the most superior in preventing infection and symptomatic infection. Whereas the inactivated vaccines were the most efficacious in preventing COVID‐19 severe infection.

With reference to COVID‐19 vaccine safety, we examined the safety in regard to local, systemic and unsolicited side effects. V‐01, COH04S1, and Sinopharm vaccines exhibited the highest safety profile in local side effects. Pfizer, QazCovid‐in, and Clover vaccines on the other hand, showed the highest risk of developing local side effects compared to placebo. Meanwhile, ZyCov‐D, V591, V‐01, and Sinopharm vaccines proved to be the safest in systemic side effects; however, Pfizer, Clover, and QazCovid‐in vaccines demonstrated the worst risk of developing systemic side effects. Furthermore, with respect to unsolicited side effects, ZF2001 vaccine presented the highest safety with KNCOV, Coronavac, and Sinopharm vaccines coming at a close second, third, and fourth, respectively. Whereas IMBCAMS vaccine disclosed the lowest safety with a 183% risk of developing unsolicited side effects. It is important to mention that Sinopharm uses an alum adjuvant, which has been widely used in several other vaccine types on the market, arguably one of the most reactogenic adjuvants. However, safety is of utmost importance and was intricately observed for the development of vaccine‐associated enhanced respiratory disease (VAERD) and adverse drug events (ADE). There was no evidence of these events in the extended follow‐ups or the ongoing Phase III trial. In addition, many of the other COVID‐19 vaccines under development also utilize the alum adjuvant with no cases of VAERD. On the contrary, alum may in fact reduce immunopathology in comparison to nonadjuvanted COVID‐19 vaccines. , In the comparison between COVID‐19 vaccines type in local side effects, protein subunit‐based vaccines closely followed by inactivated vaccines had the best safety profile while mRNA‐based vaccines had the worst safety profile. Furthermore, the comparison between COVID‐19 vaccines types in systemic side effects revealed that inactivated vaccines were the safest. Similarly, in the comparison in terms of unsolicited side effects, inactivated vaccines had the best safety profile while mRNA vaccines had the worst.

On the same note, certain adverse events had been reported after J&J, Astrazeneca, Pfizer, and Moderna vaccines administration, namely thromboembolic events and cases of capillary leak syndrome and myocarditis. However, our study showed that only 16 thrombotic events occurred and no cases of myocarditis or capillary leak syndrome had been reported in the clinical trials of Astrazeneca, J&J, Pfizer, and Moderna. Observational studies showed that after the administration of the first dose of Astrazeneca vaccine, it was found to be associated with a small increased risk of Immune Thrombotic Thrombocytopenic Purpura indicating a higher risk of the mentioned event. In addition, cases of Cranial Venous Thrombosis (CVST) had been reported after the administration of J&J vaccine. Similarly, in an article published by Sangli et al., catastrophic thrombosis was described after the administration of the second dose of the SARS‐CoV‐2 messenger RNA (mRNA)−1273 vaccine from Moderna. Thus far, no confirmed cases of Vaccine Induced Thrombotic Thrombocytopenia (VITT) had been reported after either of the mRNA vaccines despite administration in the United States alone of more than 110 million doses of the Moderna vaccine and 135 million doses of the Pfizer–BioNTech mRNA vaccine (BNT162b2). It was of great magnitude to ensure post licensure surveillance; however, we must keep in mind that it was yet illogical to establish a link between this fatal thrombotic event and the Moderna vaccine from 1 case report among the hundreds of millions of vaccine doses administered. In a systematic review by Sharifian‐Dorche et al., two of the included articles displayed 13 patients with Cranial Venous Thrombosis (CVST) and VITT after J&J vaccine. Furthermore, several studies showed a link between Pfizer and Moderna vaccines and myocarditis cases. Nevertheless, it is paramount to mention that these were extremely rare complications and studies showed that the rate of thromboembolic event and myocarditis after COVID‐19 infection was significantly higher than after taking COVID‐19 vaccine. , As a result, this must not encumber citizens not belonging to high‐risk groups of such complications to refuse the opportunity of obtaining prophylactic vaccination against a potentially fatal infection. Correspondingly, regardless of which vaccine is used, it is clear that the risk of developing postvaccination thrombocytopenia is much lower than the risk of death and morbidity from SARS‐CoV‐2 infections.

When it comes to safety and efficacy in the elderly population, it is important to identify which vaccine suits this demographic best as they tend to be vulnerable to more severe forms of COVID‐19 infection and subsequent complications. First, the most effective vaccine against COVID‐19 infection in the elderly was Pfizer at 95% while the least effective was Curevac. In fact, a cohort study demonstrated 64% efficacy 7 days after the second dose of Pfizer was administered to Long Term Care Facility (LTCF) residents which is a promising real‐world result considering this is arguably the most vulnerable group of the population where immunosenescence in the elderly population is generally a challenge for vaccines. Most vaccines documented lower efficacy amongst the elderly population. Correspondingly, similar results were seen where influenza vaccines showed lower efficacy in the older population. As a matter of fact, in a study investigating influenza vaccine efficacy, a single dose of ChAdOx1 did not durably maintain T‐cell responses above prevaccination levels in elderly individuals. On the other hand, in the comparison between COVID‐19 vaccines types among elderly, viral vector vaccines were the most superior in preventing COVID‐19 infection. Second, the safest vaccine in regard to local side effects among the elderly was V‐01. In contrast, Pfizer presented the least safety with a very high risk of developing local side effects. Third, V‐01 and Medicago were the safest vaccines in relation to systemic side effects with Clover and Pfizer coming in last at 131% and 150% risk, respectively. In the comparison between COVID‐19 vaccine types among elderly, inactivated vaccines were the safest in terms of local side effects while mRNA vaccines were the safest in terms of systemic side effects.

Some studies investigated the efficacy of their vaccines against the B.1.351 variant. Upon our analysis, we found low efficacy of both Astrazeneca and Novavax vaccines at 10% and 42%, respectively, against the B.1.351 variant. In addition to this, J&J vaccine also mentioned in a conference that the protection its vaccine provided was consistent across all variants and regions studied, including South Africa where the main offender was the B.1.351 variant, in 95% of infections. After some in vitro studies showed significant reduction in the neutralizing ability of Moderna and Pfizer vaccine against the B.1.351 lineage, several companies are testing modifications of their vaccines in animal studies to cope with the emergence of the SARS‐CoV‐2 VOCs with early promising results.

After analyzing data from the 39 included clinical trials, we came to the conclusion of an array of recommendations. To begin with, we recommend the use of Pfizer against symptomatic COVID‐19 infection. For patients likely to develop severe COVID‐19 infection like immunocompromised individuals and patients with multiple comorbidities, we recommend the use of Sputnik vaccine as it was the most effective, however it can be replaced with Sinopharm or Moderna if unavailable as they showed very close efficacy. In patients where safety is of significant concern, we recommend the use of the Sinopharm vaccine as it was the vaccine with the least side effects in Phase III trials. For instance, we recommend the lastly mentioned vaccine for patients who are more prone to develop local side effects, patients with a history of high reactogenicity to other vaccines and health care workers to ensure the maintenance and ongoing presence of the health care services during this crucial time in the pandemic. In the elderly population it is important to balance between the safety and efficacy for each individual. Owing to the fact that our study revealed that Pfizer was the most effective among elderly while it was the worst in terms of safety profile. Thus, we recommend tailoring the appropriate vaccine for each person individually. However, in light of regulatory and health policy obstacles hindering accessibility of vaccines, it is important to note that not all the countries have the availability of all of the mentioned vaccines but we suggest relying on our results in choosing the most effective and the safest available vaccine in the country for each population group is crucial. Furthermore, we recommend supporting the COVAX program which helps low income countries to get their vaccines. In addition, high‐income countries, WHO and other global human organizations are recommended to support low income countries to get more access to COVID‐19 vaccines to achieve global immunity and to allow the use of individualized vaccination manners that we recommended in our study.

This network meta‐analysis has several strengths. As no trials currently compare the safety and efficacy of the various COVID‐19 vaccines directly or indirectly, this network meta‐analysis tackles an important evidence gap by comparing the available vaccines using valid meta‐analysis methods providing valuable information to clinicians and policy makers. We followed international guidelines on the conduct and reporting of systematic reviews and network meta‐analyses, including the Cochrane Handbook and PRISMA statements. Also, we were able to conduct several subanalyses for different subgroups of the population which will aid in recommending the appropriate vaccine for each individual.

This study has several limitations. Foremost, the included studies might have several types of biases such as randomization process and deviations from the intended intervention. Also, several of the planned analyses were not conducted due to the lack of data in the included studies. For example, subanalyses, which accommodate HIV patients, cancer patients, and other subgroups, could not be carried out. In addition, due to the lack of data about some vaccine types we were not able to compare all COVID‐19 vaccines types across all the outcomes. Furthermore, the included studies were conducted at different timelines and so included different variants of COVID‐19. Some studies measured the efficacy of the vaccine after seven days from second dose administration, others after 14 days. Thus, the variability of the time period after vaccine administration in the different studies creates a limitation in terms of comparing the effectiveness of the vaccines. Similarly, the safety profile of the vaccines was monitored until seven days after the second dose hence limiting the ability to assess the safety over longer periods of follow‐up. Additionally, we were not able to assess the efficacy of the included vaccines in terms of antibody responses and neutralization tests because different trials used different methods in assessing these outcomes. Moreover, we were not able to include the EMBASE database in our search, as it was not accessible through our institution library; however, this might not affect the reliability of our results, since EMBASE is included in CENTRAL, which was one of our search databases. Finally, we could not assess the inconsistency of the network meta‐analyses in our study as all the comparisons were from indirect evidence. As a result, we were not able to assess the certainty of the evidence of the network meta‐analysis using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) as inconsistency is a major part of this approach. Moreover, continuous updates on the published primary literature are necessary to keep the published evidence on COVID‐19 vaccines up to date. Although our research has been updated to the point of writing this manuscript, some vaccine trials are ongoing and being published frequently. According to the WHO, there are 114 and 185 vaccines in clinical and preclinical development, respectively.8 In addition, some of the included vaccine results in our studies such as Sinopharm and Baharat vaccines are just preliminary results and final results can be published any time. Therefore, results are liable to change as new evidence is being presented every day. Consequently, we recommend that ongoing and future trials take into account the significance of longer periods of follow‐up and include categories of the population that are more heterogeneous such as patients with chronic kidney diseases, Human Immunodeficiency Virus (HIV) patients, chronic liver disease patients and to provide data about of the outcomes among these population groups. In addition, the funnel plots for COVID‐19 vaccines safety and efficacy showed asymmetry which implies the presence of publication bias. However, it is important to mention that no method for assessing publication bias has proven to be accurate or appropriate in network meta‐analysis.

In conclusion, according to our knowledge, this is the first network meta‐analysis that compared between COVID‐19 vaccines in terms of efficacy and safety. The most important finding is that almost all of the vaccines crossed the threshold of 50% efficacy, which is the target by the WHO for any vaccine to be considered effective. However, some of them did not reach the previously mentioned threshold against the B.1.351 variant while the remainder have not yet investigated vaccine efficacy against this variant. Furthermore, each vaccine has its own strong and weak points. Thereupon, we advocate continued vaccination efforts in individualized manner that recommend the best vaccine for each group in the community, which are abundantly required to save lives and to avert the emergence of future VOCs.

FUNDING

This research did not receive any specific grant from funding agencies in the public, commercial, or not‐for‐profit sectors.

Supporting Information

Click here for additional data file.