Traditional Chinese medicine treatment for COVID-19: An overview of systematic reviews and meta-analyses.

BACKGROUND: Coronavirus disease 2019 (COVID-19) is a rapidly spreading disease that has caused an extensive burden to the world. Consequently, a large number of clinical trials have examined the efficacy of traditional Chinese medicine (TCM) for treating and preventing COVID-19, with coinciding proliferation of reviews summarizing these studies.
OBJECTIVE: This study aimed to evaluate the methodological quality and evidence quality of systematic reviews and meta-analyses on the efficacy of TCM. SEARCH STRATEGY: Seven electronic databases, including PubMed, Cochrane Library, Web of Science, China National Knowledge Infrastructure, Chongqing VIP, Wanfang Data and SinoMed, were searched for systematic reviews and meta-analyses in October 2021. Search terms such as "Chinese medicine," "Lianhua Qingwen" and "COVID-19" were used. INCLUSION CRITERIA: Systematic reviews and meta-analyses of randomized controlled trials that evaluated the efficacy of TCM treatment of COVID-19 were included. DATA EXTRACTION AND ANALYSIS: A Measurement Tool to Assess Systematic Reviews Version 2.0 (AMSTAR 2) was used to evaluate the methodological quality. The quality of evidence was graded using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system. Data extraction and analysis were performed by two reviewers independently.
RESULTS: There were 17 meta-analyses included in our overview. The intervention group was defined as TCM combined with Western medicine, while the control group was Western medicine alone. The methodological quality of all the included studies was moderate to poor. A total of 89 outcome indicators were evaluated, of which, 8 were rated as moderate quality, 39 as low quality, and 41 as very low quality. Only one outcome measure was graded as being of high quality. The moderate quality of evidence indicated that, for the treatment of COVID-19, the clinical efficacy of TCM in combination with Western medicine was better, in terms of lung recovery, rate of conversion to severe/critical cases, symptom scores, duration of symptoms, mortality, and length of hospital stay.
CONCLUSION: Evidence from the included studies shows that, compared with conventional Western medical therapy alone, the addition of TCM to COVID-19 treatment may improve clinical outcomes. Overall, the quality of evidence of TCM for COVID-19 was moderate to poor. Meta-analyses of the use of TCM in the treatment of COVID-19 can be used for clinical decision making by accounting for the experiences of clinical experts, medical policies, and other factors.

Introduction

Coronavirus disease 2019 (COVID-19) is an infectious respiratory disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Since the first report of COVID-19 in December 2019, the epidemic has rapidly swept throughout the world [1]. According to the World Health Organization, up to February 2022, there have been 404,910,528 confirmed cases of COVID-19, including 5,783,776 deaths [2].

Traditional Chinese medicine (TCM) has been widely used in the treatment of infectious diseases in China for thousands of years, including SARS, influenza, and community-acquired pneumonia [3], [4], [5], [6]. Since the outbreak of COVID-19, numerous clinical trials studying the effects of TCM in the treatment of COVID-19 have been launched. Based on the existence of these clinical trials, numerous systematic reviews and meta-analyses on the effects of TCM for COVID-19 have also been published between 2020 and 2021 [7], [8], [9], [10], [11], [12], [13], [14]. Research shows that TCM, such as Lianhua Qingwen, can significantly reduce the rate of clinical COVID-19 cases worsening to the classifications of severe or critical cases, with a risk ratio of 0.38 [15].

Despite the explosion of review literature on the use of TCM in the treatment of COVID-19, the overall efficacy of this approach was evaluated across different populations, using different compound Chinese medicines, and focusing on different outcome measures. Furthermore, few of these review studies were prepared strictly following the standards and the conclusions were limited by the quality of included trials and high heterogeneity [16], [17]. Although systematic reviews are recognized as evidence of the highest level for clinical decision making [18], the reliability of the results was greatly affected by the quality of included trials.

A Measurement Tool to Assess Systematic Reviews Version 2.0 (AMSTAR 2) is a tool for critical appraisal of systematic reviews and meta-analyses of healthcare interventions; it allows researchers to assess methodological quality and assists decision makers in the identification of high-quality systematic reviews [19]. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system is widely used to evaluate the quality of evidence, and the strength of its recommendations facilitates use by patients, clinicians, and policy makers [20].

There are now such a large number of systematic reviews and meta-analyses evaluating the clinical efficacy of TCM for the treatment of COVID-19 that identification, appraisal and consideration of each individual paper are not feasible for practitioners. Furthermore, these reviews vary in quality and scope (and include different types of preparations from the Chinese materia medica). Thus, there is a need to collate these high-level analyses and systematically evaluate their quality.

Therefore, we conducted this overview to systematically evaluate the methodological quality and quality of evidence in the published systematic reviews and meta-analyses on the use of TCM in the treatment of COVID-19, by using the AMSTAR 2 and GRADE. Through this analysis, we provide guidance for understanding the quality of current evidence, which should benefit individuals responsible for clinical decision making.

Methods

Search strategy

Seven electronic databases, including PubMed, Cochrane Library, Web of Science, China National Knowledge Infrastructure (CNKI), Chongqing VIP (VIP), Wanfang Data and SinoMed, were searched for systematic reviews and meta-analyses in October 2021. Search terms, such as “Chinese medicine,” “integrated traditional Chinese and Western medicine,” “herbal medicine,” “Lianhua Qingwen,” “Shufeng Jiedu Capsule,” “COVID-19,” “2019-nCoV,” “coronavirus,” “SARS-CoV-2,” “coronavirus pneumonia,” “systematic review” and “meta-analysis,” and their near-synonym words were used. The search included both medical subject-heading terms and text-word terms. Additional references from identified studies, reviews, unpublished work, or relevant citations provided by experts were manually checked to include potentially missed studies. Systematic reviews published from 2019-12-01 to 2021-10-31 were included. There were no restrictions on language. The full search strategy is presented in the supplementary file.

Inclusion and exclusion criteria

The inclusion criteria for this overview were as follows: (1) the study included must be a systematic review or meta-analysis of randomized controlled trials (RCTs), where systematic review was defined as a literature review that used a clearly defined search strategy in at least one electronic database to identify all studies that met pre-defined eligibility criteria along with a study selection progress; (2) the efficacy or safety of TCM (herbal decoction, patent medicine or herbal injections) for the treatment of COVID-19 was evaluated, and combined interventions using Chinese medicine and Western medicine were also eligible; (3) included patients diagnosed with COVID-19, with no restrictions on sex, age, race, occupation, course of the disease or the severity of disease; (4) the efficacy of the treatment was measured by at least one experimentally quantifiable outcome.

Articles were excluded if: (1) they were duplicate publications; (2) they were only published in conference proceedings or protocols; (3) they evaluated interventions that included TCM combined with non-drug therapies such as point thread embedding, acupuncture, Taichi and Qigong, etc.; (4) control groups also received TCM or proprietary Chinese medicine treatment.

Study selection

Search results were compiled, and duplicates were removed using EndNote X9.1 (Clarivate Analytics, Philadelphia, USA). Two reviewers independently screened study titles and abstracts retrieved from the literature search and then read the full text of studies that passed the inclusion and exclusion criteria. Any disagreements were resolved through discussion or, if necessary, with the involvement of a third reviewer.

Data extraction

Data extraction was performed independently by two reviewers using a custom data extraction form. For each systematic review or meta-analysis included, the following characteristics were extracted: first author, year of publication, intervention, control, number of included trials, total sample size, outcome measures and main conclusions. The third review author checked for accuracy and resolved any inconsistencies in the extracted data through discussion with the reviewing authors.

Assessment of methodological quality

AMSTAR 2 was used to assess the quality of the systematic reviews included by our two independent reviewers. The scale contains 16 items [18], graded as “Yes,” “Partial Yes,” and “No.” To be specific, when the reporting and implementation for an item fully met the standards, it was graded as “Yes.” “Partial Yes” was selected when the reporting and implementation of an item was insufficient. Finally, “No” was selected when there was no reporting or implementation of the item. Among the 16 items of the AMSTAR 2, seven items (2, 4, 7, 9, 11, 13 and 15) were critical to the evaluation of systematic reviews.

Criteria for rating overall confidence in the results of the systematic review by AMSTAR 2 guideline were as follows. “High” had no or one non-critical weakness; “Moderate” had more than one non-critical weakness; “Low” had one critical flaw with or without non-critical weaknesses; “Critically low” had more than one critical flaw with or without non-critical weaknesses.

Assessment of evidence quality

Furthermore, the GRADE system [19] was used to evaluate the quality of evidence for each of our chosen outcome measures. According to the five indicators of reduced quality (limitations, inconsistency, indirectness, imprecision and publication bias) and three indicators of enhanced quality (large effect size, dose–response relationship, all plausible residual confounding), we evaluated the methodological quality of each outcome synthesized with meta-analysis of included studies, and our scores were reported as “very low” (total scores < –2), “low” (total scores = –2), “moderate” (total scores = –1), or “high” (total scores = 0). Disagreements were resolved by discussion or consultation with a third author.

Results

Through a search of electronic databases, 378 articles were identified. After the exclusion of duplicate articles, 165 articles remained. After screening the titles and abstracts, we selected to read the full text of 40 articles. Finally, 17 meta-analyses were included in our overview [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37]. Most of the studies were excluded because their analysis included a combination of RCTs and other study types, or because their interventions were combined with non-drug therapies. The selection process of studies is summarized and shown in Fig. 1 .

Fig. 1

Flow diagram. CNKI: China National Knowledge Infrastructure; VIP: Chongqing VIP; RCT: randomized controlled trial.

Study characteristics

All included meta-analyses were conducted on the Chinese mainland between 2020 and 2021. Seven of the 17 articles [21], [24], [30], [31], [33], [34], [37] were published in 2020 and 10 were published in 2021 [22], [23], [25], [26], [27], [28], [29], [32], [35], [36]. Thirteen were published in English [21], [22], [23], [24], [25], [27], [28], [30], [31], [32], [34], [36], [37] and 4 were published in Chinese [26], [29], [33], [35]. One meta-analysis included both patients diagnosed with COVID-19 and patients suspected of having COVID-19 [24], while the rest of the meta-analyses included only patients with confirmed cases of COVID-19. Of the 17 reviews, 13 (13/17, 72.2%) [21], [22], [23], [24], [26], [27], [28], [29], [30], [31], [32], [34], [36] comprehensively examined the efficacy and safety of TCM without differentiating between specific formulas. Four (4/17, 23.5%) of the included reviews evaluated the effects of the Lianhua Qingwen preparation [25], [33], [35], [37]. Among the 17 included reviews, the number of included trials ranged from 2 to 25, and the number of included participants ranged from 154 to 2257. Two reviews limited the clinical classification of COVID-19 to mild and moderate, and one limited it to moderate. Study characteristics are presented in Table 1 .

Table 1

Characteristics of included systematic reviews and meta-analyses.

Study	Publication year	Clinical status of participants	Number of included trials	Total sample size	Treatment group	Control group	Risk of bias evaluation	Main conclusion
Ang et al. [21]	2020	Diagnosed	7	855	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	Significant effects of the combined therapy of herbal medicine with WM were found.
Du et al. [22]	2021	Diagnosed (mild and moderate)	12	1393	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	Chinese herbal medicine combined with conventional therapy may be effective and safe in the treatment of mild to moderate COVID-19.
Du et al. [23]	2021	Diagnosed	9	1286	TCM contains honeysuckle + WM	Conventional WM	Cochrane risk of bias assessment tool	Honeysuckle combined with conventional therapy may be beneficial for the treatment of COVID-19 in improving lung CT, clinical cure rate, clinical symptoms, and laboratory indicators, and reducing the rate of conversion to severe cases. Combination therapy did not increase adverse events.
Fan et al. [24]	2020	Diagnosed	7	732	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	TCM, as an adjunct treatment with standard care, helps to improve treatment outcomes in COVID-19 cases.
Fan et al. [25]	2021	Diagnosed and suspected (mild and moderate)	5	824	LH preparation + WM	Conventional WM	Cochrane risk of bias assessment tool	LH in combination with usual treatment may improve the clinical efficacy in patients with mild or moderate COVID-19 without increasing adverse events.
Zhou et al. [26]	2021	Diagnosed	6	470	TCM/CPM + WM	Conventional WM	Cochrane risk of bias assessment tool	Chinese herbal decoction combined with conventional WM has some advantages in relieving clinical symptoms of cough and fatigue and can shorten the hospital stay.
Li et al. [27]	2021	Diagnosed	8	750	Oral TCM/TCM injection + WM	Conventional WM	NOS/Jadad	The integration of TCM with WM significantly improves the treatment for COVID-19 patients compared to WM treatment alone.
Liang et al. [28]	2021	Diagnosed	7	1079	Oral TCM + WM	Conventional WM	Cochrane risk of bias assessment tool	Oral TCM may have add-on potential therapeutic effects for patients with non-serious COVID-19. There are some differences in therapeutic effects between different oral TCM for the same COVID-19 outcome.
Liu et al. [29]	2021	Diagnosed	7	588	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	The effectiveness of the combination of TCM and WM in treating COVID-19, in terms of total effective rate, syndrome scores, disappearance rate of clinical symptoms, lung CT, and risk of adverse effects, was better than that of the control group that received only WM.
Pang et al. [30]	2020	Diagnosed	11	1259	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	TCM may bring potential benefits to patients suffering from COVID-19. However, the quality of included trials is not good enough. High-quality studies with a core outcome set are still required.
Sun et al. [31]	2020	Diagnosed	7	681	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	TCM combined with conventional treatment was the better treatment choice for COVID-19.
Wang et al. [32]	2021	Diagnosed	25	2222	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool 2	TCM treatment plus routine care may promote a clinical cure and chest image improvement compared to routine care alone, while reducing clinical deterioration, development of ARDS, use of mechanical ventilation, and death in patients with COVID-19. TCM treatment plus routine care may not change the rate of negativity on the SARS-CoV-2 nucleic acid test, compared to routine care alone. TCM treatment was found to be safe for patients with COVID-19.
Zhang et al. [33]	2020	Diagnosed (Moderate)	5	600	LH preparation + WM	Conventional WM	Cochrane risk of bias assessment tool	LH preparation in combination with WM is effective and has few adverse effects in the treatment of patients with the moderate COVID-19.
Xiong et al. [34]	2020	Diagnosed	18	2257	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	TCM may be beneficial for the treatment of COVID-19 and appeared to improve clinical symptoms, imaging, and laboratory indicators, shorten the course of the disease, and reduce the number of severe cases.
Tang et al. [35]	2021	Diagnosed	5	824	LH preparation + WM	Conventional WM	Cochrane risk of bias assessment tool	Compared with the conventional WM, the use of LH in combination with WM can produce an intervention effect on clinical symptoms, lung CT, and inflammatory indicators, and can shorten the duration of fever. Its safety profile remains to be confirmed by further studies.
Yin et al. [36]	2021	Diagnosed	19	1853	TCM/CPM/TCM injection + WM	Conventional WM	Cochrane risk of bias assessment tool	The integrated medicine can improve the clinical symptoms, chest CT and infection indicators of COVID-19 patients.
Zeng et al. [37]	2020	Diagnosed	2	154	LH preparation + WM	Conventional WM	Cochrane risk of bias assessment tool	The treatment of new pneumonia with LH can be used as an effective therapy to improve the clinical symptoms of new coronary pneumonia.

ARDS: acute respiratory distress syndrome; COVID-19: coronavirus disease 2019; CPM: Chinese patent medicine; CT: computerized tomography; LH: Lianhua Qingwen; NOS: Newcastle–Ottawa Scale; TCM: traditional Chinese medicine; WM: Western medicine.

Assessment of methodological quality with AMSTAR 2

AMSTAR 2 was used to rate the overall confidence in the methodological quality of the meta-analyses included in this review. The methodologies of two reviews were graded as moderate, 11 as low quality, and 4 as critically low quality.

According to the recommendations of AMSTAR 2, none of the meta-analyses reported on the funding sources of the studies included in their reviews. Of the 17 reviews, 6 provided the registration number of study protocol, but the other 11 did not, which made it difficult to evaluate any inconsistencies between the protocol and the final analysis. All 17 reviews included only RCTs but they did not explain how they selected that study designs for inclusion in the analysis. All of the included reviews stated that they conducted the study screening process, and that data extraction was conducted by two independent reviewers. The Cochrane risk of bias assessment tool was used to evaluate all 17 reviews and only one study was found to have used an unsatisfactory technique for assessing the risk of bias. Nine reviews analyzed their included trials to explore the publication bias with a funnel plot. Three analyses failed to report potential conflicts of interest, including any funding they received for conducting the review. The AMSTAR 2 scoring is presented in Table 2 .

Table 2

Assessment of methodological quality by A Measurement Tool to Assess Systematic Reviews Version 2.0.

Author (year)	Item																Methodological quality
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
Ang et al. [21] (2020)	Y	Y	Y	Y	Y	Y	PY	Y	Y	N	Y	Y	Y	Y	N	Y	Low
Du et al. [22] (2021)	Y	Y	N	Y	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	Y	Y	Moderate
Du et al. [23] (2021)	Y	Y	N	Y	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	N	Y	Low
Fan et al. [24] (2020)	Y	Y	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	N	Y	Low
Fan et al. [25] (2021)	Y	N	N	Y	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	N	Y	Critical low
Zhou et al. [26] (2021)	Y	N	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	N	N	Critical low
Li et al. [27] (2021)	Y	N	N	PY	Y	Y	PY	Y	N	N	Y	Y	Y	Y	Y	Y	Low
Liang et al. [28] (2021)	Y	N	N	PY	Y	Y	PY	Y	Y	N	Y	Y	Y	Y	N	Y	Critical low
Liu et al. [29] (2021)	Y	N	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	Y	Y	Low
Pang et al. [30] (2020)	Y	Y	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	Y	Y	Moderate
Sun et al. [31] (2020)	Y	N	N	PY	Y	Y	PY	Y	Y	N	Y	Y	Y	Y	Y	Y	Low
Wang et al. [32] (2021)	Y	Y	N	Y	Y	Y	PY	Y	Y	N	Y	Y	Y	Y	N	Y	Low
Zhang et al. [33] (2020)	Y	N	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	N	N	Critical low
Xiong et al. [34] (2020)	Y	N	N	Y	Y	Y	PY	Y	Y	N	Y	Y	Y	Y	Y	Y	Low
Tang et al. [35] (2021)	Y	N	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	N	N	Critical low
Yin et al. [36] (2021)	Y	N	N	PY	Y	Y	PY	Y	Y	N	Y	Y	Y	Y	Y	Y	Low
Zeng et al. [37] (2020)	Y	N	N	PY	Y	Y	PY	PY	Y	N	Y	Y	Y	Y	Y	Y	Low

Y: yes; N: no; PY: partial yes.

Assessment of quality of evidence using GRADE

One of the 17 reviews was excluded from the GRADE evaluation because it lacked a forest plot, sample size report and appraisal of the heterogeneity of individual outcomes. A total of 89 outcome indicators were identified in the 16 included reviews. The results showed that evidence was of moderate quality for 8 outcome indicators, of low quality for 39 outcome indicators, and of very low quality for 41 outcome measures. Only one of the outcomes was deemed to have high-quality evidence. There were no indicators of enhanced quality in any of the 89 outcome indicators. Table 3 describes the quality of evidence for each outcome measure.

Table 3

Quality of evidence in the included studies by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE).

Author (year)	Outcome indicators	Number of included trials	Study limitations	Inconsistency of results	Indirectness of evidence	Imprecision	Reporting bias	Quality of evidence
Ang et al. [21] (2020)	Overall efficacy	4	−1	0	0	0	−1	Low
	Rate of clinical symptom disappearance	2	−1	0	0	0	−1	Low
	Clinical symptom scores	3	−1	−2	0	−1	−1	Very low
	Laboratory indicators	4	−1	−2	0	−1	−1	Very low
	Time to viral assay conversion	5	−1	−2	0	0	−1	Very low
Du et al. [22] (2021)	Lung computerized tomography	7	−1	0	0	0	−1	Low
	Clinical cure rate	5	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	9	−1	0	0	0	−1	Low
	Rate of viral assay conversion	4	−1	−1	0	−1	−1	Very low
	Rate of clinical symptom disappearance	3	−1	−2	0	−1	−1	Very low
	Clinical symptom scores	4	−1	−2	0	−1	−1	Very low
	Laboratory indicators	6	−1	−2	0	−1	−1	Very low
	Adverse events	10	−1	−2	0	0	0	Very low
Du et al. [23] (2021)	Lung computerized tomography	4	−1	0	0	0	−1	Low
	Clinical cure rate	5	−1	0	0	0	−1	Low
	Rate of viral assay conversion	3	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	6	−1	0	0	0	−1	Low
	Rate of clinical symptom disappearance	3	−1	0	0	−1	−1	Very low
	Clinical symptom scores	3	−1	0	0	0	−1	Low
	Laboratory indicators	5	−1	0	0	0	−1	Low
	Adverse events	5	−1	0	0	0	−1	Low
Fan et al. [24] (2020)	Clinical symptom scores	3	−1	−2	0	−1	−1	Very low
	Laboratory indicators	5	−1	−2	0	−1	−1	Very low
	Lung computerized tomography	4	−1	0	0	−1	−1	Very low
Fan et al. [25] (2021)	Overall efficacy	5	−1	0	0	0	−1	Low
	Rate of viral assay conversion	4	−1	0	0	0	−1	Low
	Lung computerized tomography	3	−1	−1	0	0	−1	Very low
	Duration of clinical symptoms	3	−1	0	0	0	−1	Low
	Adverse events	2	−1	−2	0	−1	−1	Very Low
Li et al. [27] (2021)	Overall efficacy	3	−1	0	0	0	0	Moderate
	Lung computerized tomography	4	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	3	−1	0	0	0	0	Moderate
	Rate of clinical symptom disappearance	5	−1	−1	0	0	−1	Very low
	Duration of clinical symptoms	5	−1	−2	0	−1	−1	Very low
Liang et al. [28] (2021)	Clinical cure rate	2	−2	0	0	0	−1	Very low
	Rate of conversion to severe/critical cases	6	−2	0	0	−1	−1	Very low
	Rate of clinical symptom disappearance	2	−2	−1	0	−1	−1	Very low
	Lung computerized tomography	4	−2	0	0	0	−1	Very low
Liu et al. [29] (2021)	Overall efficacy	3	−1	0	0	−1	−1	Very low
	Clinical symptom scores	2	−1	−1	0	0	−1	Very low
	Rate of clinical symptom disappearance	2	−1	0	0	0	−1	Low
	Lung computerized tomography	2	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	4	−1	0	0	0	−1	Low
	Adverse events	3	−1	0	0	−1	−1	Very low
Pang et al. [30] (2020)	Rate of conversion to severe/critical cases	8	−1	0	0	0	−1	Low
	Mortality rate	2	−1	0	0	0	−1	Low
	Adverse events	8	−1	−2	0	−1	−1	Very low
	Clinical symptom scores	2	−1	−2	0	0	−1	Very low
	Rate of clinical symptom disappearance	3	−1	−2	0	0	−1	Very low
	Duration of clinical symptoms	2	−1	−2	0	0	−1	Very low
Sun et al. [31] (2020)	Overall efficacy	2	−1	0	0	0	−1	Low
	Adverse events	7	−1	−1	0	0	−1	Very low
	Rate of viral assay conversion	7	−1	−1	0	−2	−1	Very low
	Lung computerized tomography	3	−1	0	0	0	−1	Low
	Laboratory indicators	5	−1	−2	0	−1	−1	Very low
Wang et al. [32] 2021	Clinical cure rate	2	0	0	0	0	−1	Moderate
	Rate of viral assay conversion	2	0	0	0	−1	−1	Low
	Rate of conversion to severe/critical cases	3	0	0	0	0	−1	Moderate
	The incidence of clinical exacerbation	3	0	0	0	0	−1	Moderate
	Lung computerized tomography	3	0	0	0	0	−1	Moderate
	Mortality rate	3	0	0	0	0	−1	Moderate
	Adverse events	17	−1	0	0	0	−1	Low
Zhang et al. [33] (2020)	Rate of clinical symptom disappearance	3	−1	0	0	0	−1	Low
Xiong et al. [34] (2020)	Lung computerized tomography	13	−1	−1	0	0	−1	Very low
	Mortality rate	4	−1	0	0	−1	−1	Very low
	Clinical cure rate	7	−1	0	0	0	−1	Low
	Rate of conversion to mild cases	2	−1	0	0	−1	−1	Very low
	Rate of conversion to severe/critical cases	11	−1	0	0	0	−1	Low
	The length of hospital stay	2	−1	0	0	0	−1	Low
	Clinical symptom scores	2	−1	0	0	0	−1	Low
	Rate of clinical symptom disappearance	15	−1	−2	0	0	−1	Very low
	Duration of clinical symptoms	15	−1	−2	0	0	−1	Very low
	Rate of viral assay conversion	4	−1	−1	0	0	−1	Very low
	Laboratory indicators	6	−1	−2	0	−1	−1	Very low
	Adverse events	9	−1	−1	0	0	−1	Very low
Tang et al. [35] (2021)	Duration of clinical symptoms	4	−1	0	0	0	−1	Low
	Rate of clinical symptom disappearance	3	−1	0	0	0	−1	Low
	Overall efficacy	3	−1	0	0	0	−1	Low
	Lung computerized tomography	3	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	4	−1	0	0	0	−1	Low
Yin et al. [36] (2021)	Overall efficacy	6	−1	0	0	0	−1	Low
	Rate of clinical symptom disappearance	8	−1	−1	0	0	−1	Very low
	Lung computerized tomography	9	−1	0	0	0	−1	Low
	Laboratory indicators	9	−1	−2	0	−1	−1	Very low
Zeng et al. [37] (2020)	Rate of clinical symptom disappearance	2	0	−1	0	0	0	Moderate
	Duration of clinical symptoms	2	0	0	0	0	0	High

Main outcomes

According to the GRADE guideline, high-quality evidence means that the true effect lies close to that of the estimate of the effect and findings have high confidence. For moderate quality of evidence, findings can have moderate confidence, and the true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Thus, here we summarize evidence of high and moderate quality and recommend these data for clinical use. The summary of evidence quality and results is shown in Table 4, Table 5 .

Table 4

Summary of evidence quality in the included studies by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE).

Treatment	Outcome measure	Number of included reviews	Study limitations	Inconsistency of results	Indirectness of evidence	Imprecision	Reporting bias	Quality of evidence
Traditional Chinese medicine + Western medicine vs Western medicine	Lung computerized tomography	10	0	0	0	0	−1	Moderate
	Rate of clinical symptom disappearance	9	0	−1	0	0	0	Low
	Adverse events	8	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	8	0	0	0	0	−1	Moderate
	Clinical symptom scores	7	−1	0	0	0	−1	Low
	Laboratory indicators	7	−1	0	0	0	−1	Low
	Clinical cure rate	5	0	0	0	0	−1	Moderate
	Overall efficacy	5	−1	0	0	0	0	Moderate
	Rate of viral assay conversion	5	−1	0	0	0	−1	Low
	Duration of clinical symptoms	3	−1	0	0	0	−1	Low
	Mortality rate	3	0	0	0	0	−1	Moderate
	Rate of conversion to mild cases	1	−1	0	0	−1	−1	Very low
	The incidence of clinical exacerbation	1	0	0	0	0	−1	Moderate
	The length of hospital stay	1	−1	0	0	0	−1	Low
Lianhua Qingwen + Western medicine vs Western medicine	Duration of clinical symptoms	4	0	0	0	0	0	High
	Overall efficacy	3	−1	0	0	0	−1	Low
	Rate of clinical symptom disappearance	3	0	−1	0	0	0	Moderate
	Adverse events	2	−1	0	0	0	−1	Low
	Lung computerized tomography	2	−1	0	0	0	−1	Low
	Rate of conversion to severe/critical cases	1	−1	0	0	0	−1	Low
	Rate of viral assay conversion	1	−1	0	0	0	−1	Low

Table 5

Studies with high-quality and moderate-quality results.

Author (year)	Treatment	Outcome indicator	Effect size, [95% CI], and P-value (if available)	Total participants in both groups	Number of included trials	Quality of evidence
Li et al. [27] (2021)	TCM + WM vs WM	Overall efficacy	OR 2.50 [1.46, 4.29]	100/73	3	Moderate
		Rate of conversion to severe/critical cases	OR 0.35 [0.18, 0.69]	196/130	3	Moderate
Wang et al. [32] (2021)	TCM + WM vs WM	Clinical cure rate	RR 1.20, [1.04, 1.38], P = 0.01	173/173	2	Moderate
		Rate of conversion to severe/critical cases	RR 0.39, [0.18, 0.86], P = 0.02	208/206	3	Moderate
		The incidence of clinical exacerbation	RR 0.30, [0.12, 0.77], P = 0.01	81/65	3	Moderate
		Lung computerized tomography	RR 1.22, [1.07, 1.39], P = 0.01	313/314	3	Moderate
		Mortality rate	RR 0.28, [0.09, 0.84], P = 0.02	241/241	3	Moderate
Zeng et al. [37] (2020)	LH + WM vs WM	Rate of clinical symptom disappearance	OR 3.34, [2.06, 5.44], P < 0.001	72/72	2	Moderate
		Duration of clinical symptoms	OR − 1.04, [−1.60, −0.49], P < 0.001	72/72	2	High

TCM: traditional Chinese medicine; WM: Western medicine; LH: Lianhua Qingwen; CI: confidence interval; RR: relative risk; OR: odds ratio.

Lung computerized tomography

Twelve pieces of literature quantitatively analyzed the recovery of the lungs based on data from computerized tomography (CT) scans. Moderate-quality evidence suggested that TCM combined with Western medicine significantly improved the recovery rate of lung CT in patients diagnosed with COVID-19.

Disappearance rate of clinical symptoms

Twelve reviews reported the rate of recovery from clinical symptoms, including fever, cough, and fatigue. Moderate-quality evidence indicated that Lianhua Qingwen in combination with Western medicine significantly enhanced the rate of recovery from clinical symptoms.

Duration of clinical symptoms

Seven reviews reported the duration of clinical symptoms, including fever, cough, and fatigue. High-quality evidence indicated that Lianhua Qingwen in combination with Western medicine significantly shortened the duration of clinical symptoms.

Rate of conversion to severe/critical cases

The results of the rate of conversion to severe/critical cases were pooled in nine meta-analyses. The synthesized results showed that the application of Chinese medicine could help to reduce the rate of conversion to severe/critical cases, with moderate-quality evidence.

Clinical cure rate

The clinical cure rate was reported in five studies. Moderate-quality evidence suggested that TCM plus routine care could increase the clinical cure rate better than routine care alone.

Overall efficacy

Three studies assessed the overall efficacy of the Chinese medical treatment for COVID-19. Pooling of results from these studies showed that patients treated with combined TCM and Western medicine had an overall better effect. The level of evidence was moderate.

Mortality rate

Cases of death were reported in three systematic reviews. Moderate-quality evidence showed that, compared with routine care alone, TCM with routine care could decrease the death rate.

Risk of clinical exacerbation

Incidence of unfavorable clinical events, such as acute respiratory distress syndrome and mechanical ventilation, was analyzed in one systematic review. Moderate quality of evidence suggested that adjuvant treatment of Chinese medicine to Western medicine could decrease the incidence of unfavorable clinical events better than Western medicine alone.

Discussion

With the rapid transmission and worldwide spread of COVID-19 since 2019, researchers around the world have sought information from COVID-19 trials from pathophysiological basics to treatment and vaccines [38], [39], [40], [41]. Thus, a large number of clinical trials, systematic reviews and meta-analyses were generated. In the field of evidence-based medicine, data from systematic reviews based on RCTs are generally considered the highest level of information [42]. However, through literature screening, it was found that some methodological weaknesses should be noted in the published systematic reviews and meta-analyses.

Therefore, the present review uses the AMSTAR 2 scale and GRADE system to evaluate the methodological quality, quality of evidence and strength of recommendations of the included meta-analyses. This work describes the general characteristics, methodological quality, and quality of evidence of 17 meta-analyses that investigated the use of TCM in the treatment of COVID-19.

Summary of evidence

A total of 17 systematic reviews were identified in this overview. Moderate-quality evidence showed that the combination of Chinese medicine and Western medicine could help to enhance the clinical efficacy of COVID-19 treatment, mainly in lung CT, recovery from clinical symptoms, duration of clinical symptoms, rate of conversion to severe/critical cases, clinical cure rate, overall efficacy, mortality rate and risk of clinical exacerbation.

For reviews that evaluated the efficacy of Lianhua Qingwen, high-quality evidence suggested that taking the Lianhua Qingwen preparation in conjunction with conventional Western medicine could shorten the duration of clinical COVID-19 symptoms. Further, Lianhua Qingwen improved the rate of recovery from clinical symptoms, which was supported by moderate-quality evidence.

Methodological quality of included meta-analyses

We found that 89% of included studies were of low or critically low methodological quality. Systematic reviews and meta-analyses of high quality are still needed going forward. The following problems existed in the rating of methodological quality. (1) Most of the studies did not register their reviews or analyses or failed to provide the registration information in the article. This was the main reason for the lower quality of evidence. Protocols and registration are of vital importance to increase rigor and transparency of the systematic review literature [43]. (2) Common omissions in the reviews were lack of testing for publication bias and not searching grey literature. Relevant studies that had not been published yet were excluded from our research, which might have effect on the results of the meta-analyses [44]. (3) All reviews included only RCTs, but the justification for using this study design was not commonly explained. Reviews of RCTs offer the highest level of evidence, but the reason for selecting only RCTs is recommended in the AMSTAR 2 system. (4) The sources of funding for the studies included in the review were not reported in all reviews, making it impossible to judge whether the final result is objective [45].

Quality of evidence

Most of the outcome indicators were graded as being of low or very low quality. Within all the degraded factors, study limitation was the main factor. In this overview, limitations of the included systematic reviews mainly reflected the unclear risk of bias. The total risk of bias of most clinical trials was categorized as unclear since some of the items were not reported. On the other hand, some of the systematic reviews were conducted in the early stage of the COVID-19 pandemic and were limited by the number of included trials. The risk of bias was not considered in the process of quantitative synthesis, but most reviews account for the risk of bias in individual studies when interpreting the results of the review.

Moreover, high statistical heterogeneity accounted for the degradation of the inconsistency of results. Twenty-three percent of the included meta-analyses focused on an individual Chinese medical formula (Lianhua Qingwen), and the other studies synthesized data from different Chinese medicines together. Furthermore, only 18% of meta-analyses set inclusion criteria for the clinical stage of COVID-19 patients, including patients with mild to moderate symptoms. These factors might all contribute to the high heterogeneity and inconsistency of results.

Adequate investigation of publication bias was not carried out for most of the included reviews, and its potential impact on the results of the review was also ignored.

Recommendations for the future reviews

Based on the results of this overview, we found that the main problems of the included reviews were methodological. Registration and publication of protocols are important in development of systematic reviews, but many of the reviews we looked at neglected this step. It is recommended that authors register their reviews on the relevant registration platform such as international prospective register of systematic reviews (PROSPERO) [46] prior to their preparation, and report the registration information according to the guidelines for systematic reviews. Further, the financial support and conflict of interest statements of the included studies were insufficient. In fact, at the time these reviews were conducted, there were already recommended reporting guidelines for systematic reviews and meta-analyses, such as the Preferred Reporting Items for Systematic reviews and Meta-Analyses checklist. Conducting systematic reviews and meta-analyses in compliance with these guidelines would help to ensure that they receive a higher-quality rating and have greater use to researchers and clinicians. With the expansion of clinical trials, we also suggested that future systematic reviews could focus on the efficacy of individual Chinese medical formulas to reduce potential inconsistencies among therapies.

Limitations

There were some limitations in this overview. We were limited by the number of published trials. Many reviews pooled the data from different Chinese medical formulas, so we also could not distinguish the effects of individual Chinese medical formulas. Our overview was conducted based on the information reported in the included analyses. Some of these analyses did not provide adequate details, and it was difficult to determine whether the reviews were designed and conducted well.

Conclusion

Our overview shows that, compared with the use of conventional Western medicine alone, the addition of TCM, such as Lianhua Qingwen and Chinese herbal compounds, may improve the clinical efficacy of COVID-19 treatment and decrease the risk of unfavorable clinical events. Overall, the quality of evidence supporting the use of TCM in the treatment of COVID-19 was not very high. It is still necessary to conduct high-quality systematic reviews.

Funding

This work was supported by the Key Research and Development Projects from the Department of Science and Technology of Zhejiang Province (No. 2020C03126), and the Health Commission of Zhejiang Province (No. 2017KY502), China.