A Quantitative Benefit-Risk Analysis of ChAdOx1 nCoV-19 Vaccine among People under 60 in Italy.

The Oxford-AstraZeneca ChAdOx1 nCoV-19 is a vaccine against the COVID-19 infection that was granted a conditional marketing authorization by the European Commission in January 2021. However, following a report from the Pharmacovigilance Risk Assessment Committee (PRAC) of European Medicines Agency, which reported an association with thrombo-embolic events (TEE), in particular disseminated intravascular coagulation (DIC) and cerebral venous sinus thrombosis (CVST), many European countries either limited it to individuals older than 55-60 years or suspended its use. We used publicly available data to carry out a quantitative benefit-risk analysis of the vaccine among people under 60 in Italy. Specifically, we used data from PRAC, Eudravigilance and ECDC to estimate the excess number of deaths for TEE, DIC and CVST expected in vaccine users, stratified by age groups. We then used data from the National Institute of Health to calculate age-specific COVID-19 mortality rates in Italy. Preventable deaths were calculated assuming a 72% vaccine efficacy over an eight-month period. Finally, the benefit-risk ratio of ChAdOx1 nCoV-19 vaccination was calculated as the ratio of preventable COVID-19 deaths to vaccine-related deaths, using Monte-Carlo simulations. We found that among subjects aged 20-29 years the benefit-risk (B-R) ratio was not clearly favorable (0.70; 95% Uncertainty Interval (UI): 0.27-2.11). However, in the other age groups the benefits of vaccination largely exceeded the risks (for age 30-49, B-R ratio: 22.9: 95%UI: 10.1-186.4). For age 50-59, B-R ratio: 1577.1: 95%UI: 1176.9-2121.5). Although many countries have limited the use of the ChAdOx1 nCoV-19 vaccine, the benefits of using this vaccine clearly outweigh the risks in people older than 30 years. Study limitations included risk of underreporting and that we did not provide age-specific estimates. The use of this vaccine should be a strategic and fundamental part of the immunization campaign considering its safety and efficacy in preventing COVID-19 and its complications.

1. Introduction

The Oxford-AstraZeneca ChAdOx1 nCov-19 vaccine is a recombinant chimpanzee adenoviral vector encoding the SARS-CoV-2 spike glycoprotein [1,2]. Trial evidence suggests a 72% efficacy in preventing the COVID-19 infection [3]. Vaccine marketing authorization was provided by the European Medicine Agency (EMA) on 29 January 2021 [4], while the World Health Organization (WHO) approved its use on 15 February 2021 after it was evaluated by WHO’s Strategic Advisory Group of Experts on Immunization (SAGE) [5]. Following this evaluation, this two-dose vaccination was initially recommended for individuals over the age of 18 [6]. Majority of the European countries decided to administer it mainly for those under the age of 60 with some small differences between countries [4].

In March 2021, the use of ChAdOx1 nCoV-19 vaccine was paused in a number of European countries due to the reports of thromboembolic events (TEE), in particular involving unusual locations such as cerebral venous sinus and splanchnic circulation [7]. According to the first reports, these rare events tend to appear within two weeks after administration of the vaccine [4].

Even if the EMA has not changed since then the indications for the recommended age groups to receive the vaccine [4], many countries, including Italy, currently recommend the use of vaccination only for subjects over 60 years, without prejudice to the need to give the second dose to younger subjects who have already received the first dose and have not presented adverse events. As of today (8 May 2021), this decision is having obvious negative consequences for the immunization campaign, already slowed down by problems with the supply of all the approved vaccines, especially in those European countries which have been impacted the most by the pandemic.

The EMA has recently conducted a preliminary benefit–risk analysis of the ChAdOx1 nCov-19 vaccine in the adult population using data from European Union Member countries to compare the risk of TTE with the expected vaccination benefits in preventing COVID-19 and its complications [8]. Results suggests that benefits outweigh risks in individual aged at least 40 years, but not in younger people. However, we argue that the benefit–risk ratio might be even more favorable for those countries, such as Italy, characterized by a large circulation of the virus and high case fatality rate. For this reason, we used real-world data from Italy to carry out a comprehensive, country specific, benefit–risk analysis ChAdOx1 nCov-19 vaccine in different age groups.

2. Methods

2.1. Parameters of the Simulation

The benefit–risk analysis of the vaccination was carried out using Monte Carlo simulations, which is a robust method to assess uncertainty in quantitative benefit–risk analyses [9,10]. The benefit–risk analysis was conducted estimating for individuals vaccinated with ChAdOx1 nCov-19 vaccine the COVID-19 deaths prevented per excess death for TEE. For the analyses we did not only consider TEE in general, but also specific sub-types, namely disseminated intravascular coagulation (DIC) and cerebral venous sinus thrombosis (CVST). The Monte Carlo simulation was based on different parameters that we estimated from publicly available data and that are described hereafter (Table 1).

Table 1

Parameters used for Monte Carlo simulations.

	Original Estimates		Updated Estimates
Parameter	Estimate (95%CI)	Ref.	Estimate (95%CI)	Ref.
Vaccine Efficacy	72.30% (63.10–79.30%)	[3]
COVID-19 death rate (20–29)	1.37 (0.94–1.94)	[11,12]
COVID-19 death rate (30–49)	10.20 (9.40–11.10)	[11,12]
COVID-19 death rate (50–59)	55.70 (53.20–58.10)	[11,12]
CVST incidence rate (20–29)	0.64 (0.29–1.42)	[4]
CVST incidence rate (30–49)	1.81 (1.29–2.32)	[4]
CVST incidence rate (50–59)	1.00 (0.62–1.61)	[4]
DIC incidence rate (20–29)	0.60 (0.34–1.06)	[4]
DIC incidence rate (30–49)	1.08 (0.26–1.90)	[4]
DIC incidence rate (50–59)	3.07 (2.50–3.77)	[4]
TEE incidence rate (20–29)	40.14 (33.75–47.46)	[4]
TEE incidence rate (30–49)	85.08 (79.40–91.12)	[4]
TEE incidence rate (50–59)	200.73 (189.54–212.56)	[4]
RR CVST in vaccine users (20–29)	21.80 (0.28–121.30)	[4]	44.92 (18.05–92.55)	[4,13,14]
RR CVST in vaccine users (30–49)	3.67 (1.90–6.40)	[4]	7.98 (5.56–11.11)	[4,13,14]
RR CVST in vaccine users (50–59)	1.40 (0.20–5.10)	[4]	8.33 (4.31–14.56)	[4,13,14]
RR DIC in vaccine users (20–29)	23.30 (0.30–129.40)	[4]	6.84 (0.17–38.1)	[4,13,14]
RR DIC in vaccine users (30–49)	2.02 (0.54–5.16)	[4]	4.14 (2.07–7.42)	[4,13,14]
RR DIC in vaccine users (50–59)	0.23 (0.01–1.27)	[4]	0.68 (0.14–1.98)	[4,13,14]
RR TEE in vaccine users (20–29)	3.82 (1.91–6.84)	[4]
RR TEE in vaccine users (30–49)	1.30 (1.03–1.62)	[4]
RR TEE in vaccine users (50–59)	0.41 (0.29–0.56)	[4]
DIC fatality rate	0.57 (0.18–0.90)	[4]	0.53 (0.26–0.79)	[4,13,14]
CVST fatality rate	0.33 (0.13–0.59)	[4]	0.28 (0.16–0.42)	[4,13,14]
TEE fatality rate	0.22 (0.17–0.29)	[4]

DIC: Disseminated Intravascular Coagulation; CVST: Cerebral Venous Sinus Thrombosis; TEE: Thrombo-Embolic.

Vaccine efficacy: we assumed a 72% efficacy of the vaccination, as reported by Emary and colleagues [3]. As studies suggest that COVID-19 vaccine-induced immunity is at least six months [15], in our calculation we conservatively assumed vaccination being effective for eight months.

COVID-19 mortality rates: we used weekly reports from the Italian National Health Institute to obtain the number of COVID-19 deaths that occurred over the last five months (from 25 November 2020 to 13 April 2021) [11,12]. The age structure of the Italian population was obtained by the database of the National Institute of Statistics (ISTAT) [16]. Age-specific mortality rates and their 95% confidence intervals were estimated assuming a Poisson distribution of the events.

Baseline Incidence rates of TEE, DIC, and CVST:

Age-specific rates in the general population were retrieved by the PRAC report and associated documentation [4,13].

Relative risks of TEE/DIC/CVST in vaccine users: Age-specific relative risks of different adverse thrombotic events during the 14 days following the vaccination were retrieved by the PRAC report [4].

TEE/DIC/CVST fatality rates: fatality rates (and their 95% CI) of the different adverse thrombotic events were computed using the number of deaths over the total number of cases reported in the PRAC report and assuming a binomial distribution of events [4].

2.2. Updated Values of the Parameters

As the data reported in the PRAC report included only adverse events that occurred among ChAdOx1 nCov-19 vaccine users until March 16th, we conducted a new search using the Eudravigilance database [14] on 28 April. Including in the analyses the new data up to 28 April increased the number of identified cases of DIC and CVST from 7 and 18 to 27 and 116, respectively. The increased number of events allowed us to obtain more precise estimates of vaccine-associated relative risks and fatality rates for these conditions. Data on DIC were obtained searching in the section “Blood and lymphatic system disorders’’ for reported suspected reactions associated with the words “disseminated intravascular coagulation”, while CVST data were obtained searching suspected reactions in the section “Nervous system disorders’’ associated with the words “cerebral venous sinus thrombosis”. Data on the number of vaccinated subjects in the EEA countries (necessary to estimate the incidence rates of adverse events among vaccine users) were obtained from the European (ECDC) database [17]. For some countries (Cyprus, Germany, France, Liechtenstein, the Netherlands, Norway, Romania, and Slovakia), only the overall data of vaccinated subjects, without subdivision by age group, were available. In this case, the number of subjects in different age groups were calculated using the age distribution of vaccinated reported in the other EEA countries. Also, the cut-offs of age groups used in the ECDC website were slightly different from those used in the PRAC report. For this reason, we applied the 20–29 and 30–49 years baseline rates of DIC and CVST to the rates of the 18–24 years and 25–49 years age groups of vaccinated subjects to estimate the corresponding relative risks.

The benefit–risk ratio of ChAdOx1 nCoV-19 vaccination was calculated as the ratio between preventable COVID-19 deaths and vaccine-related deaths. We compared the results considering all TEE events and those restricted only to DIC and CVST (these two conditions being apparently more strongly associated to ChAdOx1 nCov-19 vaccine [4]). The analysis restricted to DIC and CVST was also re-run using the updated values of the parameters. Finally, we conducted a secondary analysis assuming different degrees of under-reporting of TEE.

To evaluate the uncertainty of our estimates, in each simulation we sampled the values of the parameters from their confidence intervals. We generated 95% uncertainty intervals (95%UI) by taking the 2.5% and 97.5% percentile estimates from 100,000 simulations. We also plotted the expected benefits and risks of each simulation to better evaluate the safety profile of the vaccine (probabilistic sensitivity analysis). All statistical analyses were performed using Stata version 15 (StataCorp, College Station, TX, USA) and the code is available upon request from the authors.

2.3. Ethics Committee Approval

The study was based on publicly available aggregate data. No ethics committee approval was necessary.

3. Results

The results of the benefit–risk assessment of ChAdOx1 nCoV-19 vaccination are shown in Table 2. Using data based on the PRAC report, we found that among subjects aged 20–29 years the benefit–risk ratio was not favorable, neither considering TEE (0.70; 95%UI 0.27–2.11), nor DIC-CVST (0.93; 95%UI 0.06–14.3). However, in other age groups the benefits of vaccination largely exceeded the risks. In particular, in the 30–49 age group, the estimated benefit–risk ratio for TEE was 22.9 (95%UI: 10.1–186.4) while for DIC and CVST was 52.2 (95%UI: 16.6–179.1). In the 50–59 age group the benefit–risk ratio was even larger (for TEE 1577.1; 95%UI: 1176.9–2121.5; for DIC and CVST 3506.7; 95%UI: 166.0–54,427.9). These results were confirmed in the analysis based on the updated data, that clearly showed a benefit for vaccine users over age 24 (Table 2).

Table 2

Estimated benefit–risk [B-R] ratio of ChAdOx1 nCoV-19 vaccination.

COVID-19 Deaths Prevented per Excess Death for Embolic and Thrombotic Events		COVID-19 Deaths Prevented per Excess Death for Disseminated Intravascular Coagulation and Cerebral Venous Sinus Thrombosis
Based on Original Estimates		Based on Original Estimates		Based on Updated Estimates
Age	B-R Ratio (95% UI)	Age	B-R Ratio (95% UI)	Age	B-R Ratio (95% UI)
20–29	0.70 (0.27–2.11)	20–29	0.93 (0.06–14.3)	18–24	1.48 (0.33–5.54)
30–49	22.9 (10.1–186.4)	30–49	52.2 (16.6–179.1)	25–49	22.7 (10.9–44.7)
50–59	1577.1 (1176.9–2121.5)	50–59	3506.7 (166.0–54,427.9)	50–59	305 (109.7–851.9)

A secondary analysis accounting for the under-reporting of cases shows that even assuming that a substantial proportion of cases was missed by Eudravigilance, the benefit–risk ratio would remain positive for subjects above age 30 (Table 3).

Table 3

Estimated benefit–risk [B-R] ratio of ChAdOx1 nCoV-19 vaccination, assuming different extents of under-reporting of embolic and thrombotic events.

	30% under-Reporting	50% under-Reporting	80% under-Reporting
Age	B-R Ratio (95% UI)	B-R Ratio (95% UI)	B-R Ratio (95% UI)
18–24	0.49 (0.19–1.46)	0.34 (0.13–1.04)	0.14 (0.05–0.42)
25–49	16.1 (7.10–131.6)	11.4 (5.03–98.1)	4.59 (2.03–36.5)
50–59	1104.4 (823.2–1483.4)	788.9 (588.7–1058.7)	315.6 (235.7–422.6)

Plots of the probabilistic sensitivity analysis clearly show how, in the 30–49 and 50–59 age groups, in almost all simulations the number of prevented COVID-19 deaths exceeded that of vaccine-related deaths (Figure 1, Figure 2, Figure 3, Figure 4, Figure 5 and Figure 6).

Figure 1

Probabilistic sensitivity analysis. Number of vaccine-prevented deaths for COVID-19 (benefit) vs. vaccine-related deaths for thrombo-embolic events (risk) after 100,000 simulations. Individuals aged 20–29 years, Italy.

Figure 2

Probabilistic sensitivity analysis. Number of vaccine-prevented deaths for COVID-19 (benefit) vs. vaccine-related deaths for thrombo-embolic events risk) after 100,000 simulations. Individuals aged 30–49 years, Italy.

Figure 3

Probabilistic sensitivity analysis. Number of vaccine-prevented deaths for COVID-19 (benefit) vs. vaccine-related deaths for thrombo-embolic events (risk) after 100,000 simulations. Individuals aged 50–59 years, Italy.

Figure 4

Probabilistic sensitivity analysis. Number of vaccine-prevented deaths for COVID-19 (benefit) vs. vaccine-related deaths for disseminated intravascular coagulation and cerebral venous sinus thrombosis (risk) after 100,000 simulations. Individuals aged 20–29 years, Italy.

Figure 5

Probabilistic sensitivity analysis. Number of vaccine-prevented deaths for COVID-19 (benefit) vs. vaccine-related deaths for disseminated intravascular coagulation and cerebral venous sinus thrombosis (risk) after 100,000 simulations. Individuals aged 30–49 years, Italy.

Figure 6

Probabilistic sensitivity analysis. Number of vaccine-prevented deaths for COVID-19 (benefit) vs. vaccine-related deaths for disseminated intravascular coagulation and cerebral venous sinus thrombosis (risk) after 100,000 simulations. Individuals aged 50–59 years, Italy.

4. Discussion

Although Italy, like many other European countries, has limited the use of the ChAdOx1 nCoV-19 vaccine to individuals older than 60 years, the benefits of using this vaccine clearly outweigh the risks for everybody older than 30 years.

The country is currently considering extending again the ChAdOx1 nCoV-19 vaccine to those aged 50–59 years [18]. However, the decision to restrict its use to certain age groups without a clear directive from the EMA has already impacted the immunization campaign greatly, considering that not only the decision itself but also the subsequent media coverage might have profoundly eroded public confidence and increased the risk of vaccine hesitancy [19].

The COVID-19 pandemic has had a great impact on the European Health System, since the beginning [20,21,22,23]. Italy, due to a mix of factors including the EMA delay in granting marketing authorization for all COVID-19 vaccines, shortages in vaccine supply, and logistic problems, is currently struggling with the national immunization campaign whilst experiencing the end of a third, very long COVID-19 wave. On the other hand, countries with high vaccination coverage such as the US, the UK, and Israel have seen a steady decline in COVID-19 cases and deaths, regardless of the type of COVID-19 vaccine predominantly used for the immunization campaign [24].

Although other COVID-19 vaccines have shown a higher efficacy in preventing COVID-19 than the ChAdOx1 vaccine and might be preferred by many of the countries in the long term, when vaccine supply won’t be a problem anymore, there are important factors to consider. Like other vaccines, the ChAdOx1 nCoV-19 vaccine efficacy in preventing COVID-19 complication and deaths is reported to be over 90% [25]. Furthermore, its low market price and its relatively high temperature for storage and transport (2–8 °C), can facilitate immunization even in settings with limited resources.

Our work has some limitations, the main one being represented by the possible under-reporting of the data reported in the Eudravigilance database. As adverse effects are not always reported, the real cases of thrombotic events may be higher than those detected. As shown in Table 3, however, even in the presence of substantial under-reporting, the results suggest a clear benefit in everybody aged at least 30 years. Another important limitation is related to the fact that we did not produce gender-specific estimates. Future research should evaluate whether this vaccine is associated with larger TEE risk among young women, also considering that some of them take estrogen–progestin contraceptives or hormone replacement therapy. Finally, we only focused our benefit–risk analysis on TEE deaths because ChAdOx1 nCoV-19 immunization campaign was paused in many countries due to these events being reported even if other rare vaccine-related complications (e.g., anaphylactic shock) can lead to death. However, given how rare these events are we expect this to have little impact on our estimations.

5. Conclusions

COVID-19 is having a dramatic impact on the Italian and other European health systems. The Oxford-AstraZeneca ChAdOx1 nCov-19 is a safe adenoviral vector-based vaccine with a 72% efficacy in preventing the COVID-19 infection [1,3]. This vaccine is currently recommended only for subjects over 60 years, without prejudice to the need to give the second dose to younger subjects who have already received the first dose and have not presented adverse events. However, the present analysis found that the benefits of using this vaccine clearly outweigh the risks for everybody older than 30 years. Therefore, the use of ChAdOx1 nCoV-19 vaccine, extending its recommendation to younger population as well, should be a strategic and fundamental part of the immunization campaign considering its safety and efficacy in preventing COVID-19 and its complications.