Efficacy and safety of current treatment interventions for patients with severe COVID-19 infection: A network meta-analysis of randomized controlled trials.

This study aimed to assess the efficacy and safety of different medications available at present for severe coronavirus disease 2019 (COVID-19) infection. We searched databases for randomized controlled trials (RCTs) published up to April 30, 2021, with Chinese or English language restriction, of medications recommended for patients (aged 18 years or older) with severe COVID-19 infection. We extracted data on trials and patient characteristics, and the following primary outcomes: all-cause mortality (ACM), and treatment-emergent adverse events (TEAEs). We identified 1855 abstracts and of these included 15 RCTs comprising 3073 participants through database searches and other sources. In terms of efficacy, compared with the standard of care (SOC) group, no significant decrease in ACM was found in α-lipoic acid, convalescent plasma (CP), azithromycin, tocilizumab, methylprednisolone, interferon beta, CP/SOC, high dosage sarilumab, low dosage sarilumab, remdesivir, lopinavir-ritonavir, auxora, and placebo group. Compared with placebo, we found that a significant decrease in ACM was only found in methylprednisolone (odds ratio [OR]: 0.16, 95% confidence interval [CI]: 0.03-0.75]. With respect to TEAEs, the CP group showed lower TEAEs than placebo (OR: 0.07, 95% CI: 0.01-0.58) or SOC (OR: 0.05, 95% CI: 0.01-0.42) group for the therapy of severe COVID-19 patients. This study only demonstrated that methylprednisolone was superior to placebo in treating patients with severe COVID-19 infection. Meanwhile, this further confirmed that the safety of other treatment interventions might be inferior to CP for the therapy of severe COVID-19 patients.

all‐cause mortality

α‐lipoic acid

azithromycin

confidence interval

coronavirus disease 2019

convalescent plasma

duration of study

high dosage sarilumab

interferon beta

interquartile range

lopinavir–ritonavir

low dosage sarilumab

network meta­analysis

odds ratio

randomized controlled trials

risk of reported bias

standard of care

sample size

surface under the cumulative ranking area

treatment‐emergent adverse events

World Health Organization

INTRODUCTION

Currently, coronavirus disease 2019 (COVID‐19) is the leading cause of the global burden of disease and public health, and this burden has substantially increased since 2019, largely driven by cases growth and mortality. So far, the World Health Organization (WHO) has confirmed more than 160 million cases. The mortality in patients with COVID‐19 was estimated at 2.08%. Due to its high morbidity and high impact on the COVID‐19 infection, which is one of the most challenging problems adversely affecting public health and human security worldwide.

Antiviral drugs were the most frequently used treatment and showed efficacy in patients with COVID‐19. Besides antiviral drugs, there were other pharmacological options for the treatment of COVID‐19, for example, convalescent plasma (CP), monoclonal antibody, and hormone drugs. , , However, there were many compounds that differ in efficacy and safety, and which was the “best” medication for severe COVID‐19 infection was still unclear. Due to small effect sizes in the single‐arm meta‐analysis in patients with COVID‐19, there was also a debate about the efficacy and safety of medications and some results of studies were contradictory. Most of the previous reviews mainly focused on the medications for COVID‐19 patients with all infection levels. There was a critical shortage for an analysis stratified by different infection levels (i.e., mild, moderate, and severe infection) in medications of COVID‐19. Patients with COVID‐19 differ substantially from the infection levels, and such differences should lead to individual treatment of this sensitive subgroup.

Although the network meta­analysis (NMA) has been done in previous studies, there was little published data on the network study for the medication of severe COVID‐19 infection. In addition, there have been further randomized controlled trials (RCTs) of several of the drugs since the literature search for the therapy of COVID‐19. , Some large‐scale RCTs of medications in patients with COVID‐19 have been completed. A reassessment of the available evidence to support clinical decision‐making is urgently needed.

How do we select the medication for patients with severe COVID‐19 infection in clinical practice? To fill this gap, we did a contemporary NMA of RCTs of medications in patients with severe COVID‐19 infection. The aim of this essay is to estimate all pharmacological interventions studied in terms of efficacy and safety from RCTs in patients with severe COVID‐19 infection.

MATERIALS AND METHODS

Search strategy and selection criteria

A literature search was performed using the following databases from their inception through April 30, 2021: PubMed, Elsevier Science Direct, Cochrane Library, Google Scholar, SpringerLink, MedRxiv, China National Knowledge Infrastructure, and Wanfangdata for RCTs published, with no language restrictions. We included RCTs of medications recommended in patients (aged 18 years or older) with severe COVID‐19 infection. Full search strategies were listed in Appendix S1. We extracted data on RCTs, patient and medications characteristics (Table S1).

Data were extracted via the search strategy by at least two independent investigators. We reviewed potentially relevant articles' abstracts and full‐texts for eligibility. We selected articles for the assessment according to the criterion: At least one statistical analysis of the association between severe COVID‐19 and treatments was presented and described as an estimate for efficacy and safety. Additional relevant research identified was manually retrieved.

We included the RCTs, of at least 1 week's duration, that enrolled confirmed patients (aged ≥18 years) with severe COVID‐19 infection according to the COVID‐19 laboratory diagnostics of WHO. All RCTs studies that measured the efficacy or safety between medications and severe COVID‐19 infection were considered for inclusion. Full inclusion and exclusion criteria were listed in the appendix (Appendix S2). We resolved any ambiguity through mutual discussion and consensus during selecting eligible studies.

Data extraction

Two reviewers (J. Q. J. and F. Z. J.) independently worked for data extraction and entered onto all data by using a standardized form. The main data extracted were estimates of efficacy and safety. We collected the following information: (1) basic characteristics, including author name, publication year, country/countries of origin, study design, method of COVID‐19 testing, patient population, sample size (SS), interventions, treatment medication dose, controls, control medication dose, follow‐up time, and primary outcomes; (2) primary outcomes, including all‐cause mortality (ACM) and rate of treatment‐emergent adverse events (TEAEs). We also contacted corresponding authors for breakdowns of the above data between severe COVID‐19‐infected patients and therapies if this information was not reported in the published article. One reviewer undertook the initial extraction of studies, and another reviewed the extraction. Differences were discussed, and a third investigator (C. Q. L.) was involved if consensus was not reached.

Quality assessment

At least two reviewers (C. Q. L., J. Q. J., C. J. F., and F. Z. J.) estimated the risk of bias for all study designs. We assessed the risk of bias with the Cochrane Risk‐of‐Bias Tool. We evaluated the certainty of evidence by using the Grading of Recommendations Assessment, Development, and Evaluation approach for the NMA.

Outcome measures and definitions

The primary outcomes were the ACM and TEAEs in patients with severe COVID‐19 infection, from the beginning of the intervention to the end of follow‐up. The ACM referred to the proportion of death due to any cause from treatment initiation to end of follow‐up in severe COVID‐19 patients. The TEAEs ratio reported as the proportion of any TEAEs for severe COVID‐19 infection from the beginning to the end of medications. The severe COVID‐19 infection represented cases with fever or suspected respiratory infection, plus one of the following: respiratory rate >30 breaths/min, severe respiratory distress, or SpO2 ≤ 93% on room air.

Data analysis

Network meta‐analysis

Network meta‐analyses were performed using STATA statistical software (Version 15; Stata Corporation). Binary variables (ACM and TEAEs) were analyzed using odds ratio (OR) with 95% confidence interval (CI). Additional details were reported in the appendix (Appendix S3). Statistical significance was defined as p < 0.05.

We merged simultaneously the direct evidence and indirect evidence or different indirect evidence through an NMA. In NMA, the group‐level data were used to analyze the effect of the intervention. The pooled effect size was measured by using the Z‐test. We synthesized the effect sizes of NMA using a fixed‐effect model (i.e., I 2 ≤ 50%) or a random‐effect model (i.e., I 2 > 50%). The surface under the cumulative ranking area (SUCRA) curve and mean ranks were used to rank the therapies for each outcome. Furthermore, the endpoints which lower was better would indicate Rank 1 was best and Rank N was worst from figures and vice versa.

Assessment of heterogeneity and inconsistency in the network

To reveal the disagreement of unequal evidence sources, we used statistical inconsistency to evaluate it by using local and global approaches. The node splitting method, which split evidence on a specific comparison into direct and indirect evidence, was used to assess the inconsistency of the NMA. No significant inconsistency existed in outcomes if p > 0.05. Moreover, the small‐study effect was estimated by using funnel plots in this NMA.

Risk of bias in the NMA

We estimated the risk of bias of included studies using the Cochrane Collaboration's Tool for Assessing Risk of Bias, classifying the risk of bias as high, unclear, or low (Figure 2). We used comparison­‐adjusted funnel plots to evaluate publication bias.

Figure 2

The quality for included randomized controlled trials. (A) Risk of bias summary (The yellow circle with a question mark represents “unclear risk of bias,” the red one with minus sign represents “high risk of bias” and the green one with plus sign represents “low risk of bias”). (B) Risk of bias graph

Sensitivity analysis

We hypothesized that the inclusion of various study designs and populations might contribute to heterogeneity and inconsistency. Sensitivity analyses were performed to assess the effect of our conclusions. We analyzed the data with the following restrictions: a multicenter study (MS), duration of study (DS), blinding, crossover design, SS, industry sponsorship, and risk of reported bias (RRB).

We used the netmeta package in R (version 4.0.5) to duplicate the NMA of the primary outcomes.

RESULTS

Description of included studies

The search identified 1855 citations, out of which 1753 were excluded for duplications, wrong study design or population (i.e., nonsevere COVID‐19 infection), inappropriate intervention, not outcome/drug of interest, nonclinical studies, non‐RCTs, review articles, commentaries, guidelines, and meta‐analysis by checking titles and abstracts. The full texts of 102 articles were obtained to check eligibility, in which 88 articles were excluded for nonfulfilling eligibility criteria, unable to check eligibility and duplications. Finally, 14 studies , , , , , , , , , , , , , were included in our NMA. Figure 1 shows the selection process for included studies.

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta‐Analyses flowchart for study selection. 

Fifteen RCTs including 3073 patients were included (Figure 1) and described in Table S1. The mean SS was 103 interquartile range (IQR) 62–397 in this analysis. The age of subjects was older than 18 years. The median duration of follow‐up treatment was 30 days (IQR: 28–60).

Quality appraisal

In total, included RCTs was generally good quality and the risk of bias summary was shown in Figure 2 and Table S2.

Network of evidence

In the network of connected RCTs (Figure 3), the width of the lines corresponded to the number of trials included each treatment comparison. From Figure 3 we could see that the result of this network was well connected. As shown in Figure 3A, the standard of care (SOC) was the most well‐connected treatment, with CP, CP/SOC, remdesivir, lopinavir/ritonavir (LPV/r), interferon beta (IFN‐β), tocilizumab, methylprednisolone, azithromycin (AZM), and auxora directly connected to it. Several sources of indirect evidence were available to inform comparisons between sarilumab such as high dosage sarilumab (HS) and low dosage sarilumab (LS), α‐lipoic acid (ALA), and SOC. Furthermore, CP, ALA, and sarilumab are directly connected to placebo in this network plot (Figure 3A). In Figure 3B there was also a direct connection between SOC and CP, LPV/r, IFN‐β, and AZM, or between placebo and HS or LS. Several indirect connections were available between SOC and sarilumab, placebo (Figure 3B).

Figure 3

Network plot of eligible comparisons for all‐cause mortality (A), and the TEAEs ratio (B) of medications in patients with severe COVID‐19. ALA, α‐lipoic acid; COVID‐19, coronavirus disease 2019; SOC, standard of care; TEAEs, treatment‐emergent adverse events

Efficacy outcomes

Fourteen studies , , , , , , , , , , , , , reported ACM as outcome measurement (Table S3). Compared with the SOC group, no significant decrease in ACM was found in ALA, CP, AZM, tocilizumab, methylprednisolone, IFN‐β, CP/SOC, HS, LS, remdesivir, LPV/r, auxora, and placebo group (Figure 4). Compared with the placebo group, a significant decrease in ACM was only found in methylprednisolone (OR: 0.16, 95% CI: 0.03–0.75). Unfortunately, we did not identify that there was a difference between placebo and other 12 medications for the ACM of severe COVID‐19 infection (Figure 4). Figure S1 (A: Based on the control of SOC; B: Based on the control of placebo) presented the ranking of medications based on cumulative probability plots and SUCRA. Compared with the SOC group, the ranking for severe COVID‐19 patients with the efficacy of reducing ACM from good to poor was as follows: methylprednisolone (SUCRA: 87.0%), ALA (SUCRA: 77.6%), LPV/r (SUCRA: 77.3%), auxora (SUCRA: 70.5%), LS (SUCRA: 54.4%), HS (SUCRA: 52.7%), CP/SOC (SUCRA: 52.4%), SOC (SUCRA: 50.9%), IFN‐β‐1b (SUCRA: 48.5%), AZM (SUCRA: 44.8%), tocilizumab (SUCRA: 37.6%), CP (SUCRA: 36.5%), IFN‐β‐1a (SUCRA: 29.9%), placebo (SUCRA: 23.3%), and remdesivir (SUCRA: 13.6%). Compared with placebo group, the ranking for severe COVID‐19 patients with the efficacy of reducing ACM from good to poor was as follows: methylprednisolone (SUCRA: 86.6%), LPV/r (SUCRA: 77.4%), ALA (SUCRA: 76.7%), auxora (SUCRA: 70.3%), LS (SUCRA: 53.8%), HS (SUCRA: 53.2%), CP/SOC (SUCRA: 51.2%), SOC (SUCRA: 50.7%), IFN‐β‐1b (SUCRA: 50.0%), AZM (SUCRA: 44.5%), tocilizumab (SUCRA: 37.5%), CP (SUCRA: 36.3%), IFN‐β‐1a (SUCRA: 30.2%), placebo (SUCRA: 16.3%), and remdesivir (SUCRA: 13.4%).

Figure 4

Network meta‐analyses of the relative efficacy and safety of medications for patients with severe COVID‐19 infection. ALA, α‐lipoic acid; AZM, azithromycin; CI, confidence interval; CP, convalescent plasma; COVID‐19, coronavirus disease 2019; HS, high dosage sarilumab; IFN‐β, interferon beta; LPV/r, lopinavir–ritonavir; LS, low dosage sarilumab; OR, odds ratio; SOC, standard of care

Safety outcomes

In the safety outcome, the data from seven studies, , , , , , , were merged for analysis (Table S4). The CP group showed lower TEAEs than placebo (OR: 0.07, 95% CI: 0.01–0.58) or SOC group (OR: 0.05, 95% CI: 0.01–0.42; Figure 4). However, no significant improvement in TEAEs was found in the other treatment interventions. The figure of SUCRA showed that CP had the highest cumulative probability (85.1%) becoming the best intervention in TEAEs, followed by placebo (63.0%), AZM (56.6%), LPV/r (47.7%), LS (45.9%), SOC (44.7%), IFN‐β‐1a (43.7%), and HS (39.4%) when compared with SOC; compared with the placebo group, the ranking for severe COVID‐19 patients with TEAEs from good to poor was as follows: CP (SUCRA: 85.3%), placebo (SUCRA: 61.7%), LPV/r (SUCRA: 47.8%), LS (SUCRA: 46.8%), IFN‐β‐1a (SUCRA: 44.2%), SOC (SUCRA: 44.1%), HS (SUCRA: 39.0%), and AZM (SUCRA: 31.2%), (Figure S1 [C: Based on the control of SOC; D: Based on the control of placebo]).

Evaluation of inconsistency

According to the node‐slitting analysis (Table S5), no significant inconsistency or qualitative difference was available for the ACM and TEAEs. Thus, the consistency hypothesis was accepted in this NMA.

Sensitivity analysis and publication bias

We analyzed the potential sources of heterogeneity or inconsistency by using subgroup and meta‐regression analyses. Univariable meta‐regression and subgroup analyses indicated that there were heterogeneous sources (such as DS, blinding and RRB) for the ACM (p < 0.05) (Figure 5A). Whilst the SS and RRB were the heterogeneity source of TEAEs based on the sensitivity analysis (p < 0.05; Figure 5B).

Figure 5

Meta‐regression and sensitivity analyses for the efficacy and safety of medications in patients with severe COVID‐19 infection. (A) All‐cause mortality. (B) The ratio of treatment‐emergent adverse events. CD, crossover design; COVID‐19, coronavirus disease 2019; DS, duration of study; IS, industry sponsorship; MS, multicenter study; RRB, risk of reported bias; SS, sample size

None of the funnel plots of outcomes (ACM and TEAEs) indicated a significant asymmetry (Figure S2).

DISCUSSION

To our knowledge, the NMA was lacking in evaluating the comparative effect and safety of medications in severe COVID‐19 infection. This study is based on 15 RCTs which included 3073 severe patients randomly assigned to 15 medications or SOC or placebo. The study extended the previous work that provided a reference for selecting the medication for patients with severe COVID‐19 infection in clinical practice. We only evaluated the efficacy and safety of medications for COVID‐19 infection based on randomized placebo‐controlled trials from our previous study. However, the efficacy and safety of medications based on the control of placebo group or the control of SOC group still need further clarification. In view of this, we performed this NMA study. We found that medications seemed to vary in ACM between the controls of placebo and SOC for severe COVID‐19 patients.

Efficacy of current treatment interventions

Interestingly, we only found that methylprednisolone was more efficacious than placebo for decreasing the ACM of severe COVID‐19 infection in all 15 treatment interventions of this study. This study supports evidence from previous observations. , As we know, prolonged glucocorticoid treatment is associated with improved outcomes of acute respiratory distress syndrome (ARDS). Several reports have also shown that treatment with methylprednisolone could significantly reduce the risk of death among patients with ARDS. However, the present finding seemed to be inconsistent with our previous report. Juul et al. also found that intravenous immunoglobin might reduce the ACM for COVID‐19 patients when compared with the placebo or SOC group. On the one hand, inconsistencies might be attributable to differences in study designs (such as single‐arm meta‐analysis and NMA) and/or patient characteristics (i.e., severe COVID‐19 patients and nonsevere COVID‐19 patients). On the other hand, the interpretation of this result might be limited by insufficient SS.

Regrettably, no significant difference was found in other 12 medications (i.e., CP, AZM, LS, auxora, ALA, CP/SOC, IFN‐β‐1a, IFN‐β‐1b, remdesivir, HS, LPV/r, and tocilizumab) for the ACM of severe COVID‐19 infection. This finding was contrary to previous studies which have suggested that CP, CP/SOC, remdesivir, and IFN were associated with decreased ACM in patients with COVID‐19. , , , , It is difficult to explain this result, but it might be related to the difference of participants selection in differential studies. For instance, we selected subjects with all infection levels, and were inadequate for the analyses stratified by different infection levels (i.e., nonsevere and severe infection) in medications of COVID‐19. , , Previous studies suggested that patients with COVID‐19 differ from the infection levels often leading to different outcomes of treatment. While no significant decrease in ACM was found between SOC and 14 other medications or placebo for severe COVID‐19 infection. This result seems to be inconsistent with prior NMA studies by Zhang et al., Wu et al., and Siemieniuk et al. They suggested that tocilizumab or corticosteroids might reduce the ACM compared with SOC for COVID‐19 infection. A possible explanation for this was that we compared the ACM of SOC, which existed the bias due to the differential SOC of every country (i.e., the SOC is not standardized) except for the reasons given above. , , ,

The present study raised the possibility that our findings might be beneficial to guiding the selection of drug interventions for clinicians in severe COVID‐19 patients. Of note, based on the result of meta‐regression analysis on the heterogeneity (such as DS, blinding and RRB), the present result may be needed further verification. Thus, statistical indications of clinical superiority in this network analysis required careful interpretation.

Safety of current treatment interventions

In terms of safety, we summarized the TEAEs. We only found that the CP group showed lower TEAEs than the placebo or SOC group. As mentioned in previous literature reviews, there were some adverse events for CP in treating Ebola disease. This differs from the findings presented here. However, recent meta‐analysis studies and large‐scale RCTs , , , seemed to be consistent with our findings, which identified CP therapy had some curative effect and was good safety in treating severe COVID‐19. In either case, it is noteworthy that we must fully consider its efficacy and safety when CP is used in treating severe COVID‐19 infection. Compared with placebo or SOC group, we did not find that there was a significant difference between TEAEs and other treatment interventions (i.e., AZM, HS, IFN‐β‐1a, LPV/r, and LS). This finding was consistent with a previous report of NMA. Comparison of the findings with those of other studies , , confirmed most medications of COVID‐19 might be acceptable in TEAEs. It was worth noting that clinicians might need to select treatment regimens based on the rank of safety (i.e., SUCRA) in treating severe COVID‐19. It should be reminded, however, that further studies, which reduce the impacts of SS and RRB, will need to be undertaken based on the result of sensitivity analysis.

Limitations

This study was constrained by some limitations. First, included studies might be small in this NMA, which should be considered when interpreting the findings. Second, the published data we extracted included only two types of outcomes, some important outcomes such as discharge ratio, clinical improvement ratio, and the ratio of virological cure were not analyzed. Additionally, we also did not evaluate the outcome for most laboratory indicators (i.e., studies with molecules were not included) in this NMA. The updated NMAs with laboratory indicators need to be further implemented in future studies. Thirdly, although we did our best to include all available RCTs, we cannot eliminate the possibility of missing data. Fourth, some nodes in our network included only a few trials. The SS of actual head‐to‐head trials was very small. Hence, comparative efficacy and safety between interventions was frequently based on indirect comparisons. Finally, the sensitivity analysis showed that there were several heterogeneity sources, which may conceal or exaggerate the effect size of this network analysis. Further large‐scale RCTs, which control these confounding factors, will need to be undertaken to verify our findings. Though there are still many shortcomings in our research, it is certain that the prevention and therapy of COVID‐19 is set to change for better in the future.

CONCLUSION

In conclusion, this study only demonstrated that methylprednisolone was superior to placebo in treating patients with severe COVID‐19 infection. Meanwhile, the safety of CP might be inferior to other treatment interventions for the therapy of severe COVID‐19 patients. The present NMA reported uncertain estimates on the efficacy and safety of medications in the treatment of severe COVID‐19. Maybe it is because there was inadequate evidence of a possible reduction in ACM and the absence of TEAEs. However, this study had two strengths. One was that comprehensive meta‐analysis strategy was used to reduce the risk of publication bias. The other was that the SUCRA was used to assess possibly best intervention.

Despite these limitations, to date, the present findings might represent more comprehensive meta‐analysis of the available evidence for severe COVID‐19 infection. Future guidelines and decision‐making treatment plan should consider these results for the treatment of severe COVID‐19 infection. Importantly, the government, academia and researchers should collaborate to develop more large‐scale RCTs studies and further estimate the efficacy and safety of treatment interventions on mortality, virological and clinical outcomes for different levels of infection with COVID‐19.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

AUTHOR CONTRIBUTIONS

Qinglin Cheng and Gang Zhao had full access to all of the data in the study. Qinglin Cheng takes responsibility for the integrity of the data, the accuracy of the data analysis, and the final decision to submit for publication. Qinglin Cheng, Qingjun Jia, Zijian Fang, and Junfang Chen contributed to the study concept and design. Qinglin Cheng, Gang Zhao, Qingjun Jia, Zijian Fang, and Junfang Chen contributed to data acquisition, analysis, and interpretation. Qinglin Cheng and Gang Zhao drafted the manuscript. Qinglin Cheng and Junfang Chen did a statistical analysis. Qinglin Cheng obtained funding. Qingjun Jia, Zijian Fang, and Junfang Chen contributed to the administrative, technical, or material support.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.

Supporting information.

Click here for additional data file.