Effect of tocilizumab, sarilumab, and baricitinib on mortality among patients hospitalized for COVID-19 treated with corticosteroids: a systematic review and meta-analysis.

BACKGROUND: Randomized controlled trials (RCT) established the mortality reduction by tocilizumab (Actemra), baricitinib (Olumiant), and sarilumab (Kevzara) in hospitalized COVID-19 patients. However, uncertainty remains about which treatment performs best in patients receiving corticosteroids.
OBJECTIVES: To estimate probabilities of noninferiority between baricitinib and sarilumab compared to tocilizumab in patients treated with corticosteroids. DATA SOURCES: PubMed, Embase, Cochrane Library, and MedRxiv. STUDY ELIGIBILITY CRITERIA: Eligible RCTs assigning hospitalized adults with COVID-19 treated with corticosteroids to tocilizumab or baricitinib or sarilumab versus standard of care or placebo (control).
METHODS: Reviewers independently abstracted published data and assessed study quality with the Risk of Bias 2 tool. Unpublished data, if required, were requested from authors of included studies. The outcome of interest was all-cause mortality at 28 days. PARTICIPANTS: Twenty-seven RCTs with 13 549 patients were included. Overall, the risk of bias was low. Bayesian pairwise meta-analyses were used to aggregate results of each treatment versus control. The average odds ratio for mortality was 0.78 (95% credible interval [CrI]: 0.65, 0.94) for tocilizumab; 0.78 (95% CrI: 0.56, 1.03) for baricitinib; and 0.91 (95% CrI: 0.60, 1.40) for sarilumab. The certainty of evidence (GRADE) ranged from moderate to low. Bayesian meta-regressions with multiple priors were used to estimate probabilities of noninferiority (margin of 13% greater effect by tocilizumab). Compared to tocilizumab, there were ≤94% and 90% probabilities of noninferiority with baricitinib and sarilumab, respectively.
RESULTS: All but two studies included data with only indirect evidence for the comparison of interest.
CONCLUSIONS: Among hospitalized COVID-19 treated with corticosteroids, there are high probabilities that both baricitinib and sarilumab are associated with similar mortality reductions in comparison to tocilizumab.

Introduction

Despite 2 years of treatment advances, COVID-19 remains an important cause of death, particularly in the unvaccinated, those with comorbid illness, or older adults. Major treatment advances for hospitalized patients that have demonstrated a reduction in severe complications include the antiviral remdesivir (Veklury) [1]; low-molecular-weight heparin [2]; monoclonal antibodies [3]; and antiinflammatories, including dexamethasone (Maxidex) [4], with or without interleukin (IL) receptor 6 inhibitors (e.g. tocilizumab [Actemra] [5], and sarilumab [Kevzara] [6]), or Janus kinase [Jak] 1 and 2 selective inhibitors (e.g. baricitinib [Olumiant] [7,8]).

In July 2021, a meta-analysis on IL-6 inhibitors coordinated by WHO demonstrated reduced mortality with tocilizumab in hospitalized patients with COVID-19 [5]. Notably, study findings suggested that tocilizumab acts synergistically with corticosteroids (28% fewer deaths in patients on corticosteroids), and there are high probabilities that tocilizumab reduces mortality in specific subgroups of patients [9]. Although sarilumab did not demonstrate a reduction in mortality (odds ratio 1.08 [95% CI: 0.86–1.36]) [5], it was nonetheless recommended with the same strength as tocilizumab [10].

On January 14, 2022, the WHO guidelines added a new recommendation for use of the Jak 1,2 inhibitor, baricitinib, for hospitalized patients with COVID-19, treated with corticosteroids [10]. Due to the absence of randomized controlled trials (RCTs) directly comparing IL-6 inhibitors with baricitinib when this guideline was published, the WHO recommended initiating therapy “depending on availability,” as well as “clinical and contextual factors” [10].

To better understand how these treatments compare to each other, and to guide decisions during periods of drug shortage, we sought to determine whether baricitinib and sarilumab are noninferior to tocilizumab for reducing mortality in hospitalized patients with COVID-19 who are receiving corticosteroids. We therefore conducted a systematic review and meta-analysis leveraging a Bayesian statistical framework to estimate clinically relevant noninferiority probabilities.

Methods

This report was performed according to the Preferred Items for Systematic Reviews and Meta-analyses 2020 guideline [11]. The protocol and analysis plan were preregistered in PROSPERO (CRD42022297413) and the Open Science Framework [12].

Data sources and searches

The WHO Rapid Evidence Appraisal for COVID-19 Therapies (REACT) Working Group reported a systematic review [5], conducted between October 7, 2020 to January 11, 2021, containing published and non–peer-reviewed data on tocilizumab and sarilumab trials.

We updated this systematic review, extending the search between January 11, 2021 and April 30, 2022. In addition, we conducted a systematic review for baricitinib RCTs between database inception and April 30, 2022, as baricitinib was not included in the WHO REACT Working Group systemic review [5]. For tocilizumab, baricitinib, and sarilumab RCTs, we performed electronic searches without language restriction of MEDLINE (via PubMed), Embase, Cochrane Central Register of Controlled Trials (CENTRAL), and MedRxiv. Complete search strategies can be found in the Appendix.

Study selection

Our primary research question was whether baricitinib and sarilumab are noninferior to tocilizumab in patients receiving concomitant corticosteroids. We included RCTs involving hospitalized adults with suspected or confirmed COVID-19 and included all levels of respiratory support (e.g. supplemental oxygen, noninvasive ventilation, and mechanical ventilation) and all hospitalized locations (emergency department, noncritical care units, and critical care units) at the time of randomization. We considered RCTs that directly compared tocilizumab, baricitinib, or sarilumab to the standard of care or placebo (hereafter referred to as “control treatment”). Studies that did not include any patients who received corticosteroids were excluded.

References identified from the search were imported to Rayyan [13], and duplicates were manually removed by a single reviewer (A.M.A.). Next, reviewers (A.M.A., E.G.M., J.M.B., T.C.L.) independently screened titles and abstracts in duplicate. In case of disagreement, a third reviewer (I.E.) made the final decision after discussion with other reviewers (A.M.A. and T.C.L.). Full texts of eligible studies were then retrieved and independently assessed for eligibility by two reviewers (A.M.A. and T.C.L.).

Data extraction and quality assessment

Two reviewers (A.M..A and L.T.) independently extracted the data of interest reported in WHO REACT article [5] and from full-text articles found through our updated systematic review. Discrepancies were resolved through consensus. For eligible studies with missing data, we emailed the study investigators and if there was no reply after 30 days, or if an agreement to share data could not be reached, the study was excluded.

The outcome of interest was all-cause mortality at 28 days after randomization by intention-to-treat in patients who received corticosteroids. Because they are so similar, we also included studies that reported data 29 days after randomization.

Reviewers (A.M.A., L.T., E.G.M., G.B.L.) assessed the risk of bias in the included studies using Cochrane's Risk of Bias 2 tool [14]. Two reviewers independently evaluated each study with discrepancies resolved by consensus. A third reviewer (I.E.) was involved in case disagreements were not resolved.

Multiple studies included in the WHO REACT Working Group's review have not been published (hereafter referred to as “non–peer-reviewed”) [5]. Consequently, we were not able to independently assess their risk of bias. For these, we presented the assessment shared by WHO REACT [5], which was “based on the trial protocols and flowcharts following CONSORT together with information supplied by the investigators for each trial in a standard format” [5,15].

Data synthesis and analysis

Studies included in our systematic review were pooled and analysed because of clinically acceptable between-study heterogeneity of interventions, settings, study designs, and outcome measures [16].

Meta-analyses

We estimated the crude log odds ratio (tocilizumab, baricitinib, or sarilumab versus control) for each study. We applied a continuity correction of 0.5 events in studies with no events in at least one of the treatment arms [17]. We exponentiated and presented these results as odds ratio (OR) in forest plots for ease of interpretation.

We applied a Bayesian statistical framework [18], which updates prior beliefs (e.g. external data not from the included studies) with the results found through our systematic review to form a posterior distribution. We used medians and 95% highest-density intervals (hereafter, credible intervals [CrI]) to summarize marginal posterior distributions, defined as the narrowest interval containing 95% of the probability density function [19].

For each set of studies (tocilizumab, baricitinib, or sarilumab versus control), we fitted random-effects meta-analyses [20]. Therefore, our inferences can be viewed as the average effect of tocilizumab, baricitinib, and sarilumab from these “hypothetical populations of studies” [21], as opposed to being exclusively conditioned to the included studies as in a fixed-effect model [20].

For the average effect parameter [22], we selected a prior that covers a range of plausible effect sizes, assigning limited density to unlikely values and thereby exerting little influence on the results (hereafter referred to as a weakly informative prior; the details of which are further elaborated in the Appendix) [23,24]. Upon model fitting, we calculated the posterior probabilities of any benefit (OR <1) and of meaningful clinical effect (a priori defined as OR <0.9).

For the between-study SD parameter (tau), we used an informative prior based on the predictive distribution derived from hundreds of Cochrane meta-analyses that reported all-cause mortality [25]. We also applied a post-hoc sensitivity analysis with a weakly informative prior (Half-Normal [0.5]) [26]. Details about the between-study marginal and predictive posterior distributions can be found in the Appendix.

We used contour-enhanced funnel plots to assess small study effects visually [27]. In addition, the Arcsine test for publication bias [28] was applied for meta-analyses with >10 studies [27].

The strength of the body of evidence was independently evaluated by two reviewers (I.E. and E.G.M.) using the Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) system [29].

Meta-regressions: indirect comparisons of therapy effects

Tocilizumab was the first immunomodulatory agent to demonstrate superiority for mortality versus control [5,30]. Tocilizumab has also been widely used outside of RCTs and many consider it the standard of care. Thus, we assessed baricitinib and sarilumab's noninferiority to tocilizumab with Bayesian random-effect meta-regression models to separately estimate the indirect (across-trial) comparison between baricitinib versus tocilizumab and sarilumab versus tocilizumab. We reported the comparison parameter as ratio of odds ratios (ROR, comparison drug as the numerator), whereby tocilizumab's superiority would be defined as a ROR >1. Our prespecified noninferiority margin was 1.14 [12], based on maintaining ≥50% of tocilizumab's mean effect versus control in patients treated with corticosteroids as reported by WHO REACT [5,31]. To summarize these results, we reported 95% CrIs, probabilities of baricitinib and sarilumab superiority (ROR <1), and noninferiority (ROR <1.14). In making these comparisons, we applied multiple priors for each treatment comparison to allow for different strengths of belief [32].

For the baricitinib versus tocilizumab models, we used: (1) a “vague” belief representing no real assumption about the comparison and allowing the results to directly reflect the indirect evidence; (2) a “skeptical” belief where we did not assume a difference between the agents, but we expected there would be a 95% probability that the ratio of odds ratios would fall between 0.5 and 2.0; (3) a model “Optimistic for Baricitinib [Karampitsakos et al.]” using a prior incorporating the only RCT that directly compared the two agents [33], allowing for indirect (across-trials) with direct (within-trial) evidence; and (4) a model “Optimistic for Tocilizumab [inverse Karampitsakos et al.]” using the inverse of Karampitsakos et al. [33] as the prior as a “worst-case” comparison. For the sarilumab versus tocilizumab models, we applied the same (1) “vague” and (2) “skeptical” priors while also incorporating the results from REMAP-CAP that directly compared these two drugs as priors (hereafter, referenced as (3) “Optimistic for Sarilumab [REMAP-CAP]” and (4) “Optimistic for Tocilizumab [inverse REMAP-CAP]”) [6].

Of note, when we first planned this analysis, we had prespecified different optimistic priors; however, we ultimately replaced them with direct comparisons from REMAP-CAP (sarilumab) and Karampitsakos et al. (baricitinib) when these became available during the drafting and peer review stages, respectively [6,33]. The original prespecified comparisons are included in the Appendix.

Models were fitted using Stan [34] through the R package brms (R Foundation for Statistical Computing, Vienna, Austria) [35]. Full model specifications can be found in the Appendix. Four Markov chains were implemented with an initial warm-up phase of 2000 iterations, followed by 4000 iterations. We followed the “When to worry and how to Avoid the Misuse of Bayesian Statistics” checklist for checking details about our analysis [23,36,37], confirming the convergence and adequate sampling of the models.

All analyses were conducted in R version 4.1.2. The data and code used for analysis are available from: https://github.com/arthur-albuquerque/toci_sari_bari.

Results

Overview of the trials

After deduplication, a total of 1184 records were identified via databases and registers, and 26 were assessed for eligibility. Three were excluded because there were no relevant data on patients treated with corticosteroids [[38], [39], [40]], and one trial was excluded because we were unable to contact authors to obtain the data [41]. Another 28 records were identified through the WHO REACT Working Group article. Six were excluded as they did not provide data on patients treated with corticosteroids, and another 17 were excluded because they did not provide additional data to our systematic review. In total, we included 27 studies (13 549 total patients) in our review and meta-analysis (Fig. 1 ).

Fig. 1

Preferred Items for Systematic Reviews and Meta-analyses 2020 flow diagram.

The characteristics of patients in the included trials are shown in Table 1 . For tocilizumab versus control, there were 11 published studies [[42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52]] and 5 non–peer-reviewed studies (COVIDOSE2-SS-A, ARCHITECTS, ImmCOVA, HMO-020–0224, and PreToVid), extracted from the WHO systematic review [5]. For baricitinib versus control, there were three published trials [7,8,53] and one preprint (the baricitinib arm of the RECOVERY trial, henceforth referred to as the RECOVERY Bari) [54]. Lastly, for sarilumab versus control, there were six published studies [55,42,[56], [57], [58], [59]], where one study reported two separate trials under the same protocol and article (REGENERON-P2 + P3) [58] and one non–peer-reviewed trial (SARCOVID). Further details about each trial are shown in Appendix Table 1.

Table 1

Characteristics of patients in included trials

Study	Control treatment							Experimental treatment
	Age1	Male2	Time since symptom onset1	Lab confirmed COVID-192	Simple oxygen only2	Noninvasive ventilation2	Invasive mechanical ventilation2	Agea	Maleb	Time since symptom onset 1	Lab confirmed COVID-19 2	Simple oxygen only 2	Noninvasive ventilation 2	Invasive mechanical ventilation 2
Tocilizumab
RECOVERY Toci	64.3 (55.0–73.9)	69.0	10 (7–14)	96	44.0	41.0	14	63.5 (54.2–73.6)	66.0	9 (7–13)	95	46	41.00	13.0
REMAP-CAP	61.0 (53–70)	69.8	NA	84	1.0	66.0	33	61.0 (54–71)	73.9	NA	82	<1	70.00	30.0
REMDACTA	59	66.2	8.9 (4.7)	100	6.0	83.0	11	61	61.9	8.8 (4.8)	100	7	78.00	15.0
PreToViD	66 (56–75)	67.0	NA	100	71.0	24.0	<1	67 (60–74)	67.0	NA	100	72	22.00	1.0
EMPACTA	56.0 (45–65)	57.0	9.5 (3.0)	100	64.0	28.0	0	57.0 (46–66)	60.2	10 (3.1)	100	64	26.00	0.0
COVACTA	61.5 (53.8–70.0)	70.1	11.4 (6.9)	100	31.0	27.0	38	63.0 (52.0–71.0)	69.9	12.1 (6.6)	100	27	32.00	38.0
TOCIBRAS	57.9 (46.9–69.4)	68.7	9.5 (3.0)	100	44.0	41.0	16	54.6 (44.2–70.2)	66.7	10 (3.1)	100	60	23.00	17.0
HMO-020-0224	65.8	58.8	NA	100	0.0	29.0	71	61.8	73.0	NA	100	0	43.00	57.0
COV-AID	63.3 (56.1–72.8)	73.6	10 (9–12)	93	54.0	32.0	13	62.4 (53.3–74.8)	77.8	10 (8–12)	95	48	40.00	10.0
ImmCOVA	62 (53–68)	70.4	NA	100	33.0	67.0	0	64 (56–70)	81.8	NA	100	46	55.00	0.0
CORIMUNO-TOCI-ICU	65.4(57.5–70.5)	33.0	11 (9–14)	42	0.0	12.0	31	63.2(59.4– 70.9)	33.0	11 (9–15)	49	0	13.00	36.0
CORIMUNO-TOCI-1	63.3 (57.1–72.3)	66.0	10 (8–13)	91	100.0	0.0	0	64 (57.1–74.3)	70.0	10 (7–13)	89	100	0.00	0.0
ARCHITECTS	62 (54–71)	63.6	NA	100	9.0	0.0	91	61 (46-67)	50.0	NA	100	0	0.00	100.0
COVIDOSE2-SS-A	65(55-68)	88.0	NA	100	50.0	13.0	0	65 (53–69)	60.0	NA	100	55	0.05	0.0
BACC-Bay	56.5 (44.7–67.8)	55.0	10 (7–13)	100	74.0	6.0	1	61.6 (46.4–69.7)	60.0	9 (6–13)	100	83	3.00	0.0
CORIMUNO-TOCI-DEX	63.2 (53.6–73.3)	70.0	9 (7–11)	NA	100.0	0.0	0	63.6 (52.6–73.3)	65.0	9 (7–11)	NA	100	0.00	0.0
Baricitinib
ACTT-2	55.8 (16.0)	64.3	8 (5–11)	100	53.3	21.8	11	55.0 (15.4)	61.9	8 (5–10)	100	55.9	20.00	10.5
COV-BARRIER	57.5 (13.8)	62.0	15% < 7; 85% ≥ 7 days	100	62.0	25.0	0	57.8 (14.3)	64.0	18% < 782% ≥ 7 days	100	64	24.00	0.0
COV-BARRIER 2	58.8 (15.2)	60.0	8% < 7; 88% ≥ 7 days	100	0.0	0.0	100	58.4 (12.4)	49.0	4% < 796% ≥7 days	100	0	0.00	100.0
RECOVERY Bari	57.7 (15.5)	66.0	9 (6–11)	91	9.0	25.0	38	58.5 (15.4)	2740.0	9 (6–12)	90	9	20.00	28.0
Sarilumab
CORIMUNO-SARI-1	62.8 (56.0–71.7)	78.0	10 (8-13)	87	100.0	0.0	0	61.7 (53.0–71.1)	72.0	10 (7–13)	91	100	0.00	0.0
REGENERON-P2	60 (52.0–69.0)	77.0	NA	21	31.0	26.0	43	56.5 (45.0–68.0)	71.0	NA	28	22	23.00	54.0
REGENERON-P3	61.0 (50.0–71.0)	63.3	NA	91.6	27.0	36.0	36	63.0 (53.0–72.0)	65.2	NA	89	26	33.00	40.0
REMAP-CAP	65 (53–71)	70.8	NA	84.6	0.0	88.0	12	64.5 (53–72.5)	81.3	NA	93.6	0	83.00	17.0
SARCOVID	62 (58–71)	50.0	NA	100	100.0	0.0	0	61.5 (50.5–72)	75.0	NA	100	60	20.00	0.0
SARICOR	57	67.0	NA	100	100.0	0.0	0	61.5	57.0	NA	100	100	0.00	0.0
SARTRE	58.0 (52–64)	68.6	10 (8–12)	100	100.0	0.0	0	58.8 (52–65)	77.1	9 (8–11)	100	100	0.00	0.0
Lescure et al.	60.0 (53.0–69.5)	62.5	5 (2–10)	100	76.0	13.0	11	58.0 (48.0–67.0)	64.0	7 (3–10)	100	72	15.00	12.5

Patient characteristics specific to those co-treated with corticosteroids were not systematically reported in these RCTs. Thus, this table includes characteristics to all participants randomized in these studies, and not necessarily to only patients treated with corticosteroids.

aMedian (IQR) or Mean (SD)b%.

Risk of bias in studies

The overall risk of bias for each individual study was low (Appendix Figs. 1 and 2).

Meta-analyses of baricitinib, tocilizumab, or sarilumab versus control

Regarding baricitinib, most trials were conducted prior to tocilizumab becoming standard of care; thus, most patients in baricitinib studies were not treated with tocilizumab. However, a quarter of patients included in the RECOVERY Bari trial also received tocilizumab in a nonrandomized fashion [54]. To limit our inferences to patients treated with baricitinib monotherapy, we assumed that all patients treated with tocilizumab in the RECOVERY Bari trial were also treated with corticosteroids as this represents the standard of care RECOVERY had itself established. Based on this assumption, we calculated the number of events and patients in both arms treated with corticosteroids but not treated with tocilizumab and used these for the primary analyses. We also performed a sensitivity analysis that included all patients from RECOVERY Bari who received steroids, regardless of tocilizumab use [54] (Appendix).

Fig. 2 contains the forest plots of the direct comparisons between tocilizumab, baricitinib, and sarilumab versus control treatment. The average OR for mortality when compared to control was 0.78 (95% CrI: 0.65, 0.94) for tocilizumab; 0.78 (95% CrI: 0.56, 1.03) for baricitinib; and 0.91 (95% CrI: 0.60, 1.40) for sarilumab. The posterior probabilities of any benefit (OR <1) or clinically meaningful benefit (OR <0.9) are presented in Appendix Fig. 3A. In the sensitivity analysis whereby, we included all patients from RECOVERY Bari [53] (Appendix Fig. 4), the average effect of baricitinib was 0.76 (95% CrI: 0.56, 0.98).

Fig. 2

Meta-analyses: Forest plots of Bayesian random-effect meta-analyses of tocilizumab, baricitinib, or sarilumab versus control (three separate models). Black diamonds represent median and 95% credible intervals of posterior overall results. Purple diamonds represent the 95% prediction intervals of posterior predictive distributions. The median and 95% credible intervals of the between-study SD parameter (tau) are displayed on the left bottom corner of each forest plot. RE, random effect; CrI, credible interval; PI, prediction interval. Underlying prior distributions: average effect parameter, Normal (0, 0.752); between-study standard deviation parameter, Log-Normal (-1.975, 0.672).

Between-study heterogeneity

Tocilizumab had the lowest between-study heterogeneity compared to baricitinib and sarilumab (Appendix Fig. 5). When assuming an informative between-study standard deviation prior, all three meta-analyses had low to reasonable levels of heterogeneity (Appendix Tables 2 and 3). Post-hoc sensitivity analyses with a less informative prior showed that tocilizumab's heterogeneity remained low. In contrast, baricitinib and sarilumab sensitivity analyses shifted results from low to reasonable and fairly high levels of heterogeneity primarily due to a more limited number of completed trials and smaller sample sizes.

Small study effect assessment

Visual inspection of funnel plots showed little evidence of small study effects (Appendix Fig. 6). Accordingly, the arcsine publication bias test for tocilizumab yielded a bias estimate of –0.09 (standard error: 0.38; P: 0.81).

Certainty of evidence

The certainty of the evidence across meta-analyses ranged from moderate (tocilizumab and baricitinib) to low (sarilumab), based on GRADE. For tocilizumab, we downgraded from high certainty given the wide prediction interval (i.e. moderate certainty due to inconsistency). We downgraded baricitinib from high certainty because, when restricting RECOVERY Bari to patients who did not receive tocilizumab [54], the average effect 95% CrI also includes null or trivially harmful effects (i.e. moderate certainty due to imprecision). We downgraded sarilumab from high to low because the average effect 95% CrI includes both clinically relevant benefits and harm (i.e. low certainty due to serious imprecision). Further details on our judgments of certainty and the accompanying rationale are included in Appendix Table 4.

Main study findings: indirect comparisons between baricitinib or sarilumab versus tocilizumab

Fig. 3 includes the posterior comparisons (ROR of baricitinib versus tocilizumab [left panel] and sarilumab versus tocilizumab (right panel). Table 2 contains the 95% CrIs and posterior probabilities of noninferiority (ROR <1.14) and superiority (ROR <1.00).

Fig. 3

Meta-regressions: Indirect comparisons of therapy effects: Ratio of odds ratios between tocilizumab and baricitinib (left panel) or tocilizumab and sarilumab (right panel). Colour-filled curves represent the posterior distributions. Colour-filled areas represent the posterior probability of noninferiority (Pr < 1.14), as the percentages on top of each figure. Interval bars depict the posterior median and 95% credible intervals. Solid gray lines represent underlying prior distributions. Each belief is labeled on top of each figure. Underlying prior distributions for baricitinib versus tocilizumab results: “skeptical,” normal (0, 0.3542); “optimistic for baricitinib (Karampitsakos et al.)," normal (-0.335, 0.2642); “optimistic for tocilizumab (inverse Karampitsakos et al.)," normal (0.335, 0.2642); “vague,” normal(0, 42); Underlying prior distributions for sarilumab versus tocilizumab results: “skeptical,” normal (0, 0.3542); “optimistic for sarilumab (REMAP-CAP)," normal (-0.049, 0.1182); “optimistic for tocilizumab (inverse REMAP-CAP)," normal (0.049, 0.1182); “vague,” normal (0, 42).

Table 2

Posterior credible intervals and probabilities, meta-regression analyses

Belief	ROR (95% CrI)	Probability of noninferiority, %a	Probability of superiority, %b
Baricitinib vs tocilizumab
Vague	1.01 (0.7, 1.34)	79	47
Skeptical	1.01 (0.73, 1.32)	80	46
Optimistic for baricitinib (Karampitsakos et al.)	0.92 (0.65, 1.18)	94	73
Optimistic for tocilizumab (inverse Karampitsakos et al.)	1.1 (0.83, 1.39)	62	23
Sarilumab vs tocilizumab
Vague	1.16 (0.68, 1.73)	46	25
Skeptical	1.12 (0.76, 1.57)	54	28
Optimistic for sarilumab (REMAP-CAP)	0.99 (0.81, 1.22)	90	52
Optimistic for tocilizumab (inverse REMAP-CAP)	1.08 (0.87, 1.31)	71	25

ROR, ratio of odds ratios; CrI, credible interval.

Posterior probability below the noninferiority margin (1.14 ROR).

Posterior probability <1.0 ROR.

The posterior probabilities of baricitinib being noninferior to tocilizumab ranged from 79 (vague prior) to 94% when considering direct evidence from Karampitsakos et al. [33] (Fig. 3, left panel, and Table 2). Baricitinib comparisons were only partially influenced by underlying priors, as evidenced by similar 95% CrIs and posterior probabilities across different scenarios. Regarding sarilumab versus tocilizumab, the posterior probability of sarilumab's noninferiority was 90% when including direct evidence from REMAP-CAP (6) (Fig. 3, right panel; Table 2). By contrast, without this evidence, there was only a 46% probability of noninferiority (Fig. 3, right panel, “Vague” model).

Sensitivity analyses with weaker optimistic priors showed lower probabilities of baricitinib (Appendix Fig. 7; Appendix Table 5) and sarilumab (Appendix Fig. 8; Appendix Table 6) noninferiority versus tocilizumab. The baricitinib meta-regression results were similar when all patients from the RECOVERY Bari trial were included (Appendix Table 7).

Discussion

To date, only two preprinted head-to-head trials comparing IL-6 antagonists to each other or to Jak 1-2 inhibitors for the treatment of COVID-19 have been published [6,33]. In this context, indirect comparisons using all the available data can contribute to estimating the probability of noninferiority of baricitinib or sarilumab versus tocilizumab in hospitalized patients co-treated with corticosteroids. Using data from 27 studies comprising 13 549 patients and leveraging the limited direct evidence [6,33], we found around up to 94% probability that baricitinib and up to 90% probability that sarilumab is noninferior to tocilizumab.

The results of this study have important implications for clinical practice, especially in settings where access to each of the studied therapies may be limited or where there are important cost differences. We demonstrated a high probability that baricitinib is noninferior to the standard of care, tocilizumab [5,60]. Baricitinib has some advantages over tocilizumab including a shorter half-life that may have implications for the risk of secondary infections. Disadvantages include that baricitinib is only available in pill form and thus effectiveness may be theoretically reduced if gastrointestinal absorption is a concern; that baricitinib has not been studied in severe renal failure; and that baricitinib cannot be given in pregnancy. Although the combination of baricitinib and tocilizumab looked complimentary in RECOVERY Bari [54], we believe that the open-label nature of that trial and nonrandomized use of the combination leaves room for a properly conducted RCT evaluating combination therapy given the very real risks of immunosuppression for opportunistic infections.

With respect to sarilumab, much of the interpretation depends on the weight given to the REMAP-CAP direct comparison [6]. This trial has been preprinted since June 2021 but has not yet been peer reviewed and there are important caveats. Firstly, this trial was limited to patients who were critically ill, and inferences to those outside of critical illness are by extension. Additionally, it was not clear whether patients were randomized to tocilizumab or sarilumab concurrently at each site, or whether these comparisons themselves were made indirectly via comparison with anakinra or control patients across different sites and time periods. If a patient never had the chance to be assigned to tocilizumab or sarilumab, it is unclear that their data can truly inform a direct comparison of tocilizumab to sarilumab at the level required of a nonobservational study. Nonetheless, if the direct comparison within REMAP-CAP was valid, we believe there is a high probability of noninferiority to tocilizumab.

Another recent systematic review has compared tocilizumab to sarilumab in COVID-19 patients also treated with corticosteroids [61]. They found an OR of 107 (95% CI 0.86–1.34) when comparing tocilizumab to sarilumab and concluded they were “similar.” This comparison was made in a network meta-analysis that included patients who did not receive steroids. In contrast, we restricted the analysis to those who receive steroids (modern standard of care), have prespecified a noninferiority margin, and present probabilities of noninferiority and superiority under a variety of sensitivity analyses, thereby potentially making our findings more relevant for clinical decision making.

Our study had limitations. First, five studies were non–peer-reviewed and only the WHO REACT Working Group had access to specific trial characteristics [5]; consequently, we could not directly evaluate the risk of bias of these studies. Second, as most trials only presented data on corticosteroid-treated patients as a subgroup analysis, we were unable to extract and present patient characteristics specific to those treated with corticosteroids. Third, due to the availability of single RCTs directly comparing baricitinib and sarilumab to tocilizumab, our noninferiority estimates were based principally on indirect evidence, which warranted cautious interpretation due to possible changes in control outcomes over time [62,63]. Fourth, we could not evaluate subgroups of patients based on the level of respiratory support, the degree of illness, or the presence of other co-interventions (e.g. remdesivir, anticoagulation), which could explain some between-study heterogeneity (Fig. 2). Additional causes of heterogeneity may also have included the timing of a trial within individual waves or between variants, the individual inclusion criteria, the presence of vaccinated subjects, and the use of open-label or placebo-controlled designs.

We also want to highlight several methodological strengths. An extensive and representative collection of studies was included by updating a thorough previous systematic review [5] and adding baricitinib studies. We estimated individual meta-analytic results (Fig. 2) and assessed drug efficacy differences with meta-regression (Fig. 3; Table 2). Through the Bayesian framework, probabilities of noninferiority of baricitinib and sarilumab versus tocilizumab were calculated. The robustness of the results was assessed with a variety of relevant prior distributions [32], and the calculation of actionable posterior probabilities of noninferiority in a meta-analysis context are presented herein. We chose multiple priors to span different beliefs ranging from lack of any prior knowledge (“vague”) or assuming the result will fall within a certain range (“skeptical”) to “optimistic” using the limited direct comparison data [6,33] to pessimistic (“optimistic for tocilizumab”) using the inverse of the direct comparisons. In this way, readers can have a greater sense of how the underlying assumptions influence the data and use that to inform their own conclusions.

Conclusions

Overall, our analysis suggested that both baricitinib and sarilumab could be effective alternatives to tocilizumab to reduce mortality in hospitalized COVID-19 patients concurrently treated with corticosteroids. The later comparison depends heavily on data that may only be available when the REMAP-CAP trial is peer reviewed. Ongoing comparative effectiveness research between these three drugs in the context of higher doses of corticosteroids, widespread vaccination, and with the emergence of new variants will be of utmost importance as will properly controlled studies evaluating any combination strategies.

Transparency declaration

EGM declares non–industry-funded honouraria for speaking about Covid-19 therapeutics. Other authors do not report any potential conflict of interest. This study did not receive any specific funding. Registration: PROSPERO: CRD42022297413.

Author contributions

The first author named is the lead author. The last author is the corresponding author.

AMA was responsible for conceptualization, methodology, analysis, original draft, and review. IE, LT, EGM, and GBL were responsible for methodology and review. JMB was responsible for supervision, methodology, and review. TCL was responsible for supervision, methodology, original draft, analysis, and review.