Deficiencies in the Designs and Interventions of COVID-19 Clinical Trials.

Main Text

The coronavirus disease 2019 (COVID-19) pandemic has caused an unprecedented rush of clinical investigations. However, this urgency has also triggered concerns regarding the quality of evidence being generated.1, 2, 3, 4 Particularly concerning is that the dearth of high-quality medical evidence has contributed to a lack of confidence in existing guidelines on COVID-19 therapies, the off-label use of unproven therapies by patients and clinicians, and predatory behaviors exploiting the lack of strong scientific rationale for many tested interventions. , , Crucial to the evaluation and implementation of COVID-19 interventions is the rigor, quality, and credibility of clinical trials. We assess these characteristics in a systematic analysis of all registered interventional trials on COVID-19 to date.

We identified all interventional trials registered up to May 5, 2020, from the WHO International Clinical Trials Registry and the Global Coronavirus COVID-19 Clinical Trial Tracker. Trials were manually reviewed to exclude non-interventional trials and duplicate entries. Additional trial characteristics were abstracted from individual trial registries from the European Union, Australia/New Zealand, China, India, Germany, Iran, Japan, Netherlands, Cuba, Thailand, ISRCTN, and https://www.clinicaltrials.gov/.

Characteristics of 1,029 interventional trials registered between January 23, 2020, and May 5, 2020, organized by country or region are shown in Table S1. The top countries by registered trials were China (337), US (185), and Iran (117). Most trials tested treatments (93%), planned to enroll less than 1000 patients (88%), included hospitalized patients (71.8%), and were early phase studies (58.6%). Trials were often single-center (64.4%), randomized (78%), and open-label (59.9%). Placebo comparisons were infrequent (24.2%), and trials rarely excluded prior or concurrent use of hydroxychloroquine/chloroquine (7.9%) or non-study anti-viral agents (3.6%) based on stated inclusion and exclusion criteria.

Trial characteristics differed by geography. European trials had larger enrollments, multiple sites, and more than three arms. US trials tested more prevention interventions, enrolled healthcare workers, applied blinding, used placebos, had industry sponsors, and included international sites. China and Iran trials were associated with smaller enrollments, single-center enrollment, testing of traditional medicines, and rare international collaborations. Of note, many clinical trials in China were registered prior to the spread of the pandemic beyond China and neighboring countries, likely making it difficult to involve international sites. Iran’s strained international political relations to other countries including sanctions likely affects its ability to create clinical studies including other countries.

Figures S1A and 1B show a positive and significant correlation between trials and COVID-19 infections and deaths (Spearman r, 0.70 [p < 0.001] and 0.71 [p < 0.001], respectively) across countries. However, the percent of explained variance for either correlation was poor, with infections and deaths explaining only 29.5% and 26.9%, respectively, of the variance in trials between countries. This indicates that in many areas effected by COVID-19, clinical trials are generally not accessible or disproportionately fewer than would be expected based on disease burden.

Trials examined 309 unique COVID-19 interventions, with the 10 most prevalent interventions being tested in 65% of studies (Figure S1C). Controversial or undefined interventions including hydroxychloroquine, traditional medicine, plasma products, and stem cell products were tested in a substantial number of trials (217, 130, 58, and 53 trials, respectively). ,

We assessed the strength of preclinical evidence supporting the most popular interventional drugs purported to have direct anti-viral activity by examining all related PubMed entries (Figure S1D). PubMed queries were conducted by manually searching for the intervention name and COVID-19 or SARS-CoV-2 and reviewing each entry. Hydroxychloroquine, traditional medicine, and remdesivir had more than one publication describing experiments that directly assessed in vitro or in vivo anti-viral activity. Preclinical evidence for lopinavir/ritonavir, azithromycin, and arbidol were limited to single publications.

These findings indicate that COVID-19 interventional trials have heterogenous designs, but are predominantly open-label, single-center studies regardless of geography. Thus, most studies are unlikely to be generalizable, susceptible to reporting bias, and may not be feasible due to the large planned enrollment sizes. The infrequent use of placebo for comparisons or exclusion of other commonly used COVID-19 interventions limits the interpretation of many trials as interactions between drugs in both safety and efficacy cannot be discounted, and the additive or synergistic effects of treatments cannot be distinguished. We also noted that endpoints varied widely and included diverse surrogates, which will confound the interpretation and comparison of trials.

Geographic differences in trial characteristics, the poor alignment between trial numbers and COVID-19 disease burden, and the lack of international collaborations suggest that better integration of clinical research efforts is warranted to combat a global public health crisis. In particular, given the widespread impact of the COVID-19 across nearly all continents, it is surprising that there is very little evidence of any international collaboration. This has undoubtedly contributed to many studies examining the same interventions, which is likely wasteful, particularly because the design and endpoints in many trials are generally unique and preclude comparisons. Internal collaboration should be encouraged as it may improve the quality of trials by involving additional investigators, and global participation may help address health needs across the world and improve the generalizability of results.

This analysis also highlights neglected aspects of COVID-19 research. For instance, multiplicity of interventions being tested was prevalent, as well as a lack of attention to the efficacy of non-pharmaceutical and prevention interventions. The scientific rationale of many trials appears poor, as even the most prevalent interventions purported to have anti-viral activity were infrequently associated with preclinical evidence substantiated by independent groups. Our results call for a more careful assessment of hypotheses prior to clinical testing and a shift of research efforts from theory to scientific experimentation. While there is a clear need for speediness, safe and effective therapies can only be discerned from rigorously designed and conducted clinical trials. We believe that an assessment of the quality and rationale for COVID-19 trials has important implications not just for the current pandemic, but for the next global health crisis.