Efficacy of COVID-19 vaccines: Several modes of expression should be presented in scientific publications.

Several vaccines are being developed as part of the COVID-19 pandemic. The results of clinical trials for these vaccines were published with efficacy values of more than 90%, using mainly relative risk (RR). In this paper, we decided to reanalyse the data using the different validated methods of risk expression. Using main publications, absolute risks (AR), AR reduction (ARR), number needed to treat (NNT) were calculated for five COVID-19 vaccines (tozinameran Comirnaty®, Moderna, Vaxzevria®, Janssen, and Sputnik V vaccines). AR, ARR, NNT, and RR values varied according to COVID-19 vaccines. The order of the different vaccines was not the same according to the chosen efficacy parameters. This is a further example of the need to express results of clinical trials, using not only RR, but also AR, ARR, and NNT in order to clearly present the clinical interest of drugs.

INTRODUCTION

Several vaccines are being developed as part of the COVID‐19 pandemic. The first vaccines to be marketed were the Pfizer tozinameran Comirnaty® [1] and Moderna [2] mRNA vaccines, followed by the adenovirus vaccines from AstraZeneca (Vaxzevria®) [3] and Janssen (Ad26.COV2.S) [4, 5]. Today, the Russian vaccine, another adenovirus vaccine, is not marketed in Europe or United States [6].

The results of clinical trials for these different vaccines were published in leading medical journals as well as in the media as “very active” with efficacy values of more than 90% [7]. As clinical pharmacologists, we were interested by a clear quantification of their efficacy. Thus, we decided to reanalyse the data using the different methods of risk expression.

METHODS

After the extraction of the main data from the published papers concerning the five COVID‐19 vaccines [1, 2, 3, 4, 5], we calculated different validated expressions of infection risk:

Absolute risks (AR), that is, the risks in the exposed group (patients receiving vaccine) and in the unexposed group (control);

Absolute risk reduction (ARR), that is, the arithmetic difference between the risk in the treatment group and the risk in the control group;

Number needed to treat (NNT) calculated as the reciprocal of the ARR;

Relative risk (RR) with its 95% confidence interval (CI) defined as the ratio of the cumulative incidence of the outcome in the exposed group to the cumulative incidence in the control group [8, 9].

We also calculated the same parameters for influenzae [10] and Ebola vaccines [11] as controls.

RESULTS

Table 1 shows the different evaluation parameters for the different vaccines. The order of the different vaccines was not the same according to the chosen efficacy parameters. AR values vary between 0.04 for tozinameran to 0.34 for Janssen vaccine in exposed patients. Large differences were found in ARR between 0.67 for Janssen and 1.20 for Vaxzevria® or Sputnik V vaccines. Thus, NNT varies between 83 for Vaxzevria® and Sputnik V vaccines and 149 for the Janssen vaccine. Lower NNT values were found for influenzae and Ebola vaccines. RR was between 0.05 for tozinameran and 0.34 for Janssen vaccine.

TABLE 1

Risk of infections expressed as absolute risk (AR), absolute risk reduction (ARR), number needed to treat (NNT), and relative risk (RR) with its 95% confidence interval (CI) with the four COVID‐19 vaccines (V), influenzae vaccine (V), and Ebola vaccine (V)

	Results in exposed patients	Results in control patients	AR in exposed patients	AR in control patients	ARR	NNT	RR	95% CI
Pfizer vaccine tozinameran Comirnaty®	8/21 720	162/21 728	0.04%	0.74%	0.71%	141	0.05	0.02–0.10
Moderna vaccine	11/15 210	185/15 210	0.07%	1.20%	1.13%	91	0.06	0.03–0.11
AstraZeneca vaccine	30/5807	101/5829	0.50%	1.70%	1.20%	83	0.30	0.19–0.44
Sputnik V vaccine	16/14 964	62/4902	O.10%	1.30%	1.20%	83	0.09	0.05–0.14
Janssen vaccine	66/19 306	193/19 178	0.34%	1.01%	0.67%	149	0.34	0.26–0.45
Influenzae vaccine	221/18 797	357/13 095	1.18%	2.73%	1.56%	64	O.43	0.35–0.50
Ebola vaccine	91/91 492	880/92 262	0.10%	9.50%	9.4%	11	0.11	0.08–0.11

Note: Data were extracted from previous works. , , , , , , ,

DISCUSSION

The present work aims to present the different modes of expression of the efficacy of COVID‐19 vaccines. Several measures are used to report benefits and harms of drugs or treatments. They widely differ in their clinical relevance because they do not equally reflect the benefits in clinical terms. The most widely used measures RR (or RRR) are of limited clinical interest, because they fail to take into account baseline risks and tend to exaggerate the positive results of studies. ARR and NNT are better expressions of risk from a clinical point of view [7, 8]. In this study, we did not calculate the number needed to harm (NNH) because safety data about COVID‐19 vaccines today are limited and mainly concern “nonserious” adverse events.

Authors should mention that absolute risk estimates require a time frame to be correctly understood, that is, how long patients were treated or exposed to the drug (vaccine) or observed.

Analysis of these different benefit values allows three major conclusions. First, RR and NNT values give different approaches of the COVID‐19 vaccines efficacy. Second, the magnitude of the influenzae or Ebola vaccines' effect seems to be more important than that of COVID‐19 ones. Third, no major differences seem to appear between the different COVID‐19 vaccines; even NNT values for tozinameran or Janssen vaccines are a little bit lower than those found with the three other COVID‐19 vaccines. An indirect comparison is not possible because exposed and control populations were not strictly comparable. In fact, it was not the purpose of this paper to perform an indirect comparison of the different vaccines or to discuss the interest of COVID‐19 vaccines, but only to better illustrate, from a medical point of view, the different efficacy parameters, using validated methods [8, 9].

The paper has other limitations. We used data from scientific international publications and not regulatory data packages. As underlined above, a strict comparison between the different vaccines was not possible because different evaluation criteria were used in different published papers, even if we tried to minimize this bias. Finally, limits of NNT use are well known: no account taken of adverse events or different clinical forms of the disease, lack of reliable confidence intervals, and so forth. Some authors recommend the use of ARR values instead [12]. However, to be correctly understood, AR estimates require a time frame, that is, taking into account how long patients were treated or exposed to the drug (vaccine) or observed.

In conclusion, because RR measures do not take into account baseline risks and tend to exaggerate benefits of treatments, we believe that presentation of results in large international clinical trials should also include not only RR but also AR, ARR, and NNT in order to help prescribers (but also health authorities) truly represent the clinical interest of therapeutic interventions and make the best choice for patients and society. This conclusion is true not only for drugs in general but also for vaccines.

AUTHOR CONTRIBUTIONS

JLM and AS designed the study. JLM extracted the data and performed the statistical analysis. JLM, PB, and AS analysed the data. JLM wrote the paper. The three authors reviewed the successive versions of the manuscript and approved the final version.

CONFLICT OF INTEREST

None.

ETHICS STATEMENT

The study, which only involved review of published data without direct access to patients, was performed according to international ethics guidelines.

FUNDING INFORMATION

The work was performed during the university research time of the authors. There were no funding sources. The authors certify that they have not received any funding from any institution, including personal relationships, interests, grants, employment, affiliations, patents, inventions, honoraria, consultancies, royalties, stock options/ownership, or expert testimony for the last 48 months.