Serious Concerns About the RCT Study Quality of Traditional Chinese Medicine Injections for Treating Cardiovascular Diseases and COVID-19.

Dear Editor,

Traditional Chinese medicine (TCM) injections, i.e., the injectable preparations with active substances from herbal medicines, are the major modality of TCM treatments in treating acute and critical diseases. Cardio-cerebrovascular diseases have become the largest market of TCM injections with total sales of over 10 billion dollars in 2016 [1]. However, TCM injections possess efficacy concerns and potential risks as known worldwide in terms of clinical evidence and pharmacovigilance, e.g., content inconsistency among different batches, complexity of TCM chemical constituents, obscure rationale of prescriptions, and unclear mechanism of drug action. If TCM injections were not handled by professionals, it would do harm even after some clinical trials required for drug approval. For this reason, China’s National Medical Products Administration (NMPA), formerly known as China Food and Drug Administration (CFDA), urged re-evaluation of TCM injections in 2018 [2]. We therefore conducted a re-evaluation of randomized controlled trials (RCTs) on approved TCM injections in response to the NMPA’s call to identify obvious research integrity issues. We assessed the quality of eligible RCTs on TCM injections with the Cochrane Collaboration’s risk of bias tool [3], revealing that there were common mistakes among the RCTs in use for one of their most popular indications—angina pectoris.

Out of 475 studies including 24 TCM injections eligible for our systematic review, none satisfied the basic requirements for randomization, blinding, and/or competing interest disclosure. Figure 1 shows that the overall risk of bias in the included studies was high. Only two studies (2/475, 0.42%) described specific random sequence generation as specified by the Cochrane Handbook [3]. The other (389/475, 81.89%) studies did not report any information about randomization methods. As such, only 0.42% of the included studies claimed to be “RCTs” were evident to be randomized trials.

Fig. 1

An overview of current RCT quality on TCM injections for angina pectoris. a The risk of bias assessment of included RCTs. b Evolution of reporting proportion on quality-related issues over every 2 years. *Overall quality of RCTs was assessed by the proportion of domains with low risk among all domains according to the Cochrane Collaboration’s risk of bias tool

Almost all (444/475, 93.47%) included studies did not report how blinding was performed. Nineteen studies claimed to be single-blinded, and 12 studies claimed to be double-blinded. In addition, 46.53% of included studies were reported by a single author and highly questionable.

The included studies on TCM injections inadequately reported drug safety findings. Only 53.68% (255/475) of the included studies mentioned about drug safety, while the other studies had high risks in selective reporting bias in terms of safety. There were little concerns about the safety of TCM injections in the included studies, regardless of the common knowledge that adverse drug reaction (ADR) incidence of TCM injections contributed to over 50% of total number ADRs on TCMs [4].

All the included studies on TCM injections lacked disclosure of competing interests. Again, most of the studies (454/475, 95.58%) did not declare any financial support and competing interests. It was surprising that the ethics approval committee did not detect these issues as questionable research practices. Institutional and government investigations into potential issues in research ethics would be warranted.

The included RCTs on TCM injections could not be expected to be reproducible nor reliable to support the efficacy or safety of TCM injections. While the western RCTs have achieved gradual improvements in study quality since the publication of the Consolidated Standards of Reporting Trials (CONSORT) statement [5, 6], the quality of RCTs on TCM injections in China had no improvement at all (Fig. 1). It seems that the CONSORT statement [5] had little influence on the RCTs on TCM injections conducted in China.

In addition to angina pectoris, our findings would be also relevant to the RCTs on TCM injections in treating the new coronavirus disease 2019 (COVID-19). TCMs, especially TCM injections, have been widely prescribed to treat COVID-19 in China although there has been little reliable clinical evidence on the efficacy of TCM injections in the treatment of COVID-19 [7]. Three of our re-evaluated TCM injections (i.e., Shenfu, Shenmai, and Shengmai injections) for treating angina pectoris were also recommended as COVID-19 treatments by China’s National Health Commission [8]. Thus, we also have concerns about the RCTs on TCM injections in treating COVID-19.

As of July 2020, according to our search for COVID-19 clinical trials from the Covid-19 TrialsTracker (http://covid19.trialstracker.net/), nine clinical trial protocols (Supplemental Table 1) on six TCM injections in treating COVID-19 have been registered; the primary sponsors of the two COVID-19 trials (ChiCTR2000029756 and ChiCTR2000030388) did poor RCTs on TCM injections in treating angina pectoris in terms of their CONSORT compliance. Three of the nine COVID-19 trials (ChiCTR2000029742, ChiCTR2000029756, and ChiCTR2000030117) reported details of randomization methods (Supplemental Table 1) with low risk bias as assessed according to the Cochrane Handbook [3]. We also found that none of these clinical trial protocols specified any double-blindness. Six of them (6/9, 66.67%) were open-label studies. Drug safety outcomes, such as ADRs, were planned in only one (ChiCTR2000030388). As such, we would like to raise the public concerns about these registered RCT protocols to test TCM injections in treating COVID-19. We are hoping that the COVID-19 trials will not repeat the same mistakes happened in the clinical trials on angina pectoris.

China’s NMPA and all trialists must take the challenge to improve the evidence credibility of the RCTs on TCM injections. Understanding and practicing the Good Clinical Practice (GCP) and following the CONSORT statement [5] should be the least they must do to ensure the study quality of RCTs. We also suggest that the authorities for trial registration and ethics approval of RCT protocols should require qualified clinical trialists and medical statisticians to involve in the submitted RCT protocols.

(DOCX 17 kb)