Many trials of hydroxychloroquine for SARS-CoV-2 were redundant and potentially unethical: an analysis of the NIH clinical trials registry.

OBJECTIVE: We sought to map the landscape of trials investigating hydroxychloroquine (HCQ) for SARS-CoV-2 in order to draw conclusions about how clinical trials have been conducted in the pandemic environment and offer potential regulatory recommendations. STUDY DESIGN AND
SETTING: We identified and captured data related to registered studies using HCQ to treat SARS-CoV-2 registered with the publicly available National Institutes of Health (NIH) Clinical Trials Registry between February and November 2020.
RESULTS: Between February and November 2020, 206 studies investigating HCQ in SARS-CoV-2 were registered with the NIH Clinical Trials Registry. As of November 2020, 135 studies were listed as ongoing, 22 have been completed, and 46 are either suspended or have been terminated. Reasons for suspension or termination included difficulties with patient recruitment (n = 9), emerging evidence showing a lack of benefit of HCQ (n = 7), and recommendations by regulatory boards to discontinue (n = 10).
CONCLUSION: Many clinical trials of HCQ were launched in the first months of the pandemic, and a significant proportion of them remained active as of November 2020. The medical community appears to have responded very quickly to political interest in HCQ, while responding much more slowly to the evolving medical evidence of its lack of efficacy.

Over 200 RCTs of HCQ were launched in the first months of the Covid-19 pandemic.

There were significant levels of redundancy within these trials, suggesting some were unnecessary.

Trials were slow to terminate following conclusive demonstration of HCQ's ineffectiveness.

To prevent redundancy, trials should be centrally registered and regulation should also be centralized. Decisions about RCT oversight should be based on available clinical data rather than on concepts of "equipoise." Data from completed RCTs should be made available immediately and brought to the attention of those overseeing similar trials.

Introduction

The ongoing global pandemic due to infection with SARS-CoV-2 (aka COVID-19) has highlighted a number of preexisting questions surrounding the ethics and practice of randomized clinical trials (RCTs) [1]. For example, how many clinical trials are required to sufficiently answer a given scientific question [2]? Additionally, how should ongoing and prospective trials respond to a developing knowledge base? It is widely recognized that RCTs are essential to assess the relative efficacy of treatments, but they can also be unnecessary [3], wasteful [4], and even unethical [5]. Concerns about wasteful research contrast with the widely recognized need to replicate scientific findings [6]. Therefore, ensuring that RCTs are well-justified is an important and challenging task [7], particularly in the context of a global pandemic where the drive to identify effective treatments is magnified by time pressures and political exigencies [8,9,10,11].

In early 2020, an emerging preclinical literature suggesting the potential benefits of hydroxychloroquine (HCQ) for COVID-19 provided justification for RCTs involving humans [12]. Preliminary results in the spring of 2020 provided encouragement, as did the enthusiasm of then-American President Donald Trump [13]. Multiple RCTs of HCQ were launched in a short period of time. Several of these RCTs have since reported their results, and the majority have found no evidence of benefit [14,15]. However, many trials appear to be ongoing.

In this study, we sought to map the landscape of HCQ trials for COVID-19 in order to draw conclusions about how clinical trials have been conducted in this environment, and to consider the implications of this case study for questions of research ethics, epistemology, methodology and economy.

Methods

In this study, we aim to characterize the evolution of HCQ trials (or trials in which hydroxychloroquine or chloroquine were used interchangeably) for SARS-CoV-2 by performing a cross-sectional analysis of all relevant studies registered in the National Institutes of Health Clinical Trials Registry. Our central questions were: How many RCTs of HCQ have been launched, and how many were ongoing as of November 2020? In what clinical scenarios, and at what doses, has HCQ been tested? How did trials respond to emerging evidence, for example by termination or suspension?

We identified all studies using HCQ to treat SARS-CoV-2 registered with the publicly available National Institutes of Health (NIH) Clinical Trials Registry [16] between February and November 2020. An initial of search of the Clinical Trials Registry took place on August 1, 2020 with a second review and update taking place on November 4, 2020. We kept our search broad by placing no restrictions on country of study, ethnicity, sex, or socio-economic status of study populations, or healthcare location of research. Study type was limited to randomized and nonrandomized trials and single-arm studies; observational studies or case-series were not included in our analysis.

Study details collected by our team included: study status (completed, recruiting, active, but not recruiting, not yet recruiting, and terminated/suspended), subject population (inpatient, outpatient, healthcare workers, active disease, prophylactic), study size, co-interventions, hydroxychloroquine dose (loading and maintenance), duration of therapy, and date of study registration, initiation, termination/suspension or completion. For terminated studies, we sought to determine whether reasons for study termination had been offered. When reasons were offered, we extracted specific statements and analyzed these statements thematically. (see eTable 1 in the online-only supplement for examples of categorization). An evaluation of hydroxychloroquine treatment effect was not performed as this was deemed to be outside the scope of this study. We also did not assess study quality or risk of bias as this was not relevant to our central questions. This study did not require institutional review board approval because it was based on publicly available information and involved no patient records.

Results

Number of studies, change over time, countries of origin

Between February and November 2020, 206 studies investigating HCQ in SARS-CoV-2 were registered with the NIH Clinical Trials Registry (eTable 2). Of these, 182 (90%) were randomized controlled trials, 9 (4%) were nonrandomized trials, and 12 (6%) were single arm studies with no active comparator. The majority of these studies were launched between March and May 2020 (Fig. ) with both the number of completed and suspended/terminated studies beginning to rise after May 2020. The country of study origin varied widely (eTable 3).

Fig. 1

Time series of SARS-CoV-2 associated hydroxychloroquine studies.

Treatment context, dosing regimens, patient populations

Treatment Context: Of the 135 ongoing studies, 96 (71%) aimed to recruit patients with active disease (confirmed diagnosis of SARS-CoV-2 via laboratory testing) and 39 studies (28%) focus on prophylaxis, 7 being pre-exposure and 32 postexposure.

Comparators: In 58 studies, HCQ is compared to placebo, in 120 it is compared to one other drug, and in 83 it is compared to two or more drugs. One hundred and twenty-five studies had HCQ-only arms, while 105 studies contained arms in which HCQ was used in combination with one or more other drugs including azithromycin and lopinavir/ritonavir. In 52 studies, HCQ was compared in combination with other drugs, and in 50 studies it was compared against a combination intervention. In at least 45 (28%) ongoing studies, HCQ was being used as the standard of care.

Dosing: Three studies used the nebulized form of the drug, with the remaining studies using the oral form. The majority of studies used a loading and maintenance dose of 800 and 400 mg, respectively (Table 1 , eFigure 1). However, significant variability is seen with dosing ranging from 1200mg daily [17] to 200mg weekly [18].

Table 1

Characteristics of SARS-CoV-2 associated oral hydroxychloroquine studies stratified by study status

	Completed (n = 22)	Recruiting (n = 75)	Active; not recruiting (n = 14)	Not yet recruiting (n = 46)	Suspended/Withdrawn (n = 46)
Study type (n, %)	RCT: 19 (86.4%)NRS: 3 (13.6%)	RCT: 69 (92.0%)NRS: 3 (4.0%)Single arm: 3 (4.0%)	RCT: 11 (78.6%)NRS: 1 (7.1%)Single arm: 2 (14.3%)	RCT: 42 (91.3%)NRS: 1 (2.2%)Single arm: 3 (6.5%)	RCT: 41 (89.1%)NRS: 1 (2.2%)Single arm: 4 (8.7%)
Study population (n, %)	Inpatient: 9 (40.9%)Outpatient: 12 (54.5%)HCWs: 1 (4.5%)	Inpatient: 27 (36.0%)Outpatient: 35 (46.7%)HCWs: 13 (17.3%)	Inpatient: 8 (57.1%)Outpatient: 1 (7.1%)HCWs: 5 (35.7%)	Inpatient: 15 (32.6%)Outpatient: 20 (43.5%)HCWs: 11 (23.9%)	Inpatient: 15 (32.6%)Outpatient: 26 (56.5%)HCWs: 5 (10.9%)
Treatment type (n, %)b	Active: 17 (77.3%)Prophylactic: 5 (22.7%)	Active: 60 (80.0%)Prophylactic: 15 (20.0%)	Active: 9 (64.3%)Prophylactic: 5 (35.7%)	Active: 27 (58.7%)Prophylactic: 19 (41.3%)	Active: 39 (84.7%)Prophylactic: 7 (15.2%)
Treatment arms (n, %)	HCQ monotherapy: 12 (54.5%)Combined therapy: 12 (54.5%)	HCQ monotherapy: 40 (53.3%)Combined therapy: 45 (60.0%)	HCQ monotherapy: 11 (78.6%)Combined therapy: 6 (42.9%)	HCQ monotherapy: 28 (60.9%)Combined therapy: 26 (56.5%)	HCQ monotherapy: 34 (73.9%)Combined therapy: 16 (34.7%)
Comparison against placebo (n, %)	10 (45.5%)	25 (33.3%)	7 (50.0%)	15 (32.6%)	17 (36.9%)
Represented “standard of care” (n, %)	6 (27.2%)	21 (28.0%)	1 (7.1%)	16 (34.8%)	1 (2.2%)
Loading dose,a mg; mean (SD)	673.7 (251.31)c	690.6 (314.57)d	769.2 (110.94)f	689.5 (211.54)f	746.3 (264.66)g
Dose range (mg)	400–1400	200–1600	400–800	200–1200	200–1600
Maintenance dose,a mg; mean (SD)	423.7 (147.54)c	458.0 (212.36)e	446.1 (166.41)f	463.1 (154.95)f	452.5 (108.57)h
Dose range (mg)	200–800	200–1200	200–800	200–1000	200–800
Dosing frequencyDuration, days; mean (SD)	Daily (n = 19)Weekly (n = 1)Not specified (n = 1)	Daily (n = 67)Weekly (n = 3)Not specified (n = 5)	Daily (n = 10)Weekly (n = 3)Not specified (n = 1)	Daily (n = 33)Weekly (n = 8)Not specified (n = 5)	Daily (n = 40)Weekly (n = 2)Not specified (n = 4)
Dosing duration, days; mean (SD)	Daily: 15 (20)Weekly: 30Not specified (n = 1)	Daily: 19 (24)Weekly: 66 (31_Not specified (n = 5)	Daily: 6 (1)Weekly: 54 (27)Not specified (n = 1)	Daily: 16 (32)Weekly: 85 (45)Not specified (n = 5)	Daily: 7 (3)Weekly: 135 (63)Not specified (n = 4)

Abbreviations: RCT, randomized controlled trial; NRS, nonrandomized study; HCQ, hydroxychloroquine; HCWs, healthcare workers; SD, standard deviation.

Dosing from daily and weekly regimens were pooled.

Prophylaxis was considered pre- or postexposure.

Missing 3 values.

Missing 11 values.

Missing 13 values.

Missing 8 values.

Missing 5 values.

Missing 6 values.

Patient Population: Twenty-nine (21%) studies reported recruiting healthcare workers with the remaining studies recruiting subjects from the inpatient (n = 50) and outpatient (n = 56) settings. Almost all studies were enrolling participants of both sexes, with the exception of one study that exclusively recruited male migrant workers (NCT04446104) and three studies that focused on pregnant women (NCT04410562, NCT04354441, NCT04365231). The majority of studies were recruiting adults only (n = 191, 94%); 15 studies (7%) indicated that children could be enrolled.

Terminations/suspensions/response to evidence

As of November 2020, 135 studies were listed as ongoing (75 studies recruiting, 16 studies that were active but not presently recruiting, and 46 studies that have not started recruitment). Twenty-two studies have been completed, and 46 studies were either suspended or have been terminated (see Table 1 for details). Four studies that are presently categorized as “active, but not recruiting” have published data in peer-reviewed journals (NCT04316377 [19], NCT04322123 [20], NCT04329832 [21], NCT04328467 [22]). As outlined in Table 2 , the proposed outcomes to be evaluated by the studies still recruiting were similar to the outcomes used in the 22 completed studies and include both clinical and laboratory-based outcomes. Of the 22 studies categorized as completed, 4 have been published in peer-reviewed journals (NCT04321278 [23], NCT04261517 [24], NCT04332991 [25], NCT04308668 [26,27]) with the remaining 18 studies being either unpublished at this time or releasing early results on preprint servers (NCT04343092, NCT04304053, NCT04384380).

Table 2

Comparison of completed and ongoing trials

	Completed (n = 22)	Recruiting (n = 75)
Number of patients mean (SD)	637 (1,159)	1,333 (4,882)
Peer reviewed published studies (n, %)	4 (18%)	NA
Interventions	• HCQ monotherapy • HCQ combined with one or more of: • Azithromycin • Ivermectin • Lopinavir/Ritonavir/Favipiravir • Exercise	• HCQ monotherapy • HCQ combined with one or more of: • Azithromycin • Vitamin C • Ivermectin • Bromhexine • Nitazoxanide • Emtricitabine • Toclizumab/Imantinib • Oseltamivir/Lopinavir/Ritonavir/ Favipiravir • Radiation therapy • Interferon beta-1B
Primary outcome (n)Studies treating active Disease	• Laboratory confirmed virological clearance (n = 6) • Clinical response or improvement on an ordinal scale (n = 6) • Mortality (n = 3) • Clinical resolution and virological clearance (n = 1) • Radiological improvement (n = 1)	• Laboratory confirmed virological clearance (n = 17) • Clinical response or improvement on an ordinal scale (n = 13) • Mortality or hospitalization/ICU (n = 22) • Clinical resolution and virological clearance (n = 1) • Radiological improvement (n = 1) • Dyspnea or blood oxygen saturation < 92% (n = 4) • Tolerating medication (n = 1)
Primary Outcome (n)Prophylactic Studies	• Incidence of laboratory confirmed SARS-CoV-2 (n = 5)	• Incidence of laboratory confirmed SARS-CoV-2 (n = 6) • Symptomatic SARS-CoV-2 infection rate (n = 7) • SARS-CoV-2 free survival (n = 2)

Abbreviations: HCQ, hydroxychloroquine; NA, not applicable.

Of the 46 suspended or terminated studies, 31 (67%) provided a reason or rationale for study termination/suspension (Table 3 ). The reasons vary widely and include difficulties with patient recruitment (n = 9), emerging evidence showing a lack of benefit of HCQ (n = 7), and recommendations by regulatory boards to discontinue (n = 10).

Table 3

Reasons/rationales for suspension, withdrawal, or termination of trial or hydroxychloroquine-associated trial arms (n = 46)

Study suspended (n = 10)	Trial withdrawn/terminated (n = 29)	Study arm terminated (n = 7)a
No rationale (n = 4)	No rationale (n = 5)	No rationale (n = 6)
Decision by DSMB (n = 2)	Lack of patients (n = 8)	Emerging evidence showing lack of benefit (n = 1)
Lack of patients (n = 1)	Emerging evidence showing lack of benefit (n = 8)
Emerging evidence showing lack of benefit (n = 1)	Recommendation by health regulatory board (n = 6)
Recommendation by health regulatory board (n = 1)	Concern about potential side effects (n = 1)
Concern about potential side effects (n = 1)	Lack of interest from study sites for enrollment (n = 1)

Abbreviation: DSMB, data safety monitoring board.

Studies are ongoing, Hydroxychloroquine arm is halted.

Discussion

This attempt to map the landscape of clinical trials of HCQ for COVID-19 supports three broad conclusions that pertain to clinical trial justification and oversight, especially in the context of a pandemic. We have endeavored to match each of these conclusions with operationalizable recommendations that might guide our response to any future pandemic.

The first conclusion to draw from this cross-sectional survey is that a very large number of clinical studies of HCQ for COVID-19 were launched in the first months of the pandemic, and that a significant proportion of them remained active as of November 2020. In the context of a global pandemic, the instinct to formally study potential treatments rather than adopt them without evidence should be celebrated and encouraged. However, this large number of studies suggests a certain amount of redundancy, which is relevant as a methodological, economic, and ethical problem. Many of these studies have major similarities to each other, and to already completed and published studies. There is no doubt that reproducibility is an essential component for high quality scientific research, and that having more than one study confirm any given finding adds to scientific confidence. At the same time, once a given question has been answered, a subsequent trial is unethical for a number of reasons: some patients will be randomized to receive what is already known to be an inferior treatment and hence could be harmed by their participation, [28] and clinical research that is unlikely to provide a meaningful contribution to medical knowledge is also unethical on the grounds of wastefulness [1]. While it goes beyond the scope of this study to adjudicate whether each identified study was or was not wasteful, a scenario in which 200 trials are enrolling simultaneously is unlikely to be efficient or beneficial. An additional consequence of a large number of trials dedicated to one treatment is that research resources are allocated towards these costly trials at the expense of investigating other potentially beneficial treatments [29].

To prevent these circumstances from arising again, we suggest that certain changes in the regulatory landscape be made. First, we believe our conclusions point to the benefit of centralized registration so that active and upcoming trials are easily identifiable. For example, national and regional regulatory authorities could agree to share data into a common searchable registry, so that any and all active and prospective trials would be easily identifiable. Second, they support centralized rather than local research ethics review. While the current research ethics landscape privileges national and regional authority, there is no specific reason why this must be when the underlying principles of ethics review should be generally acceptable, and are largely derived from international documents such as the Declaration of Helsinki. If all (or at least many) of the trials for HCQ had sought approval from a centralized body, then the overlap between them would have been much more apparent and could have been mitigated.

The second major conclusion of this study is that the medical community appears to have responded very quickly to political interest in HCQ, while responding much more slowly to the evolving medical evidence of its lack of efficacy. While HCQ was being used without trial evidence at the start of the pandemic, broad public interest was seemingly spurred by a series of statements from US President Donald Trump who, on March 20, 2020, publicly embraced the drug's potential, asking “What the hell do you have to lose?” [30] Prior to this statement, only 6 randomized trials of HCQ had been registered with clinicialtrials.gov; over the following month, that number rose to 97, and it has remained over 100 throughout the remainder of 2020. In comparison, the first major trials of HCQ began to publish results in June 2020, none of which identified any benefit for the drug as prevention or treatment [31,32]. While our survey did identify over 40 trials that have since been terminated or suspended, 75 trials appear to be actively recruiting with many dozens more still listed as active or ready to launch. Moreover, only 9 of the terminated studies cited the emerging evidence surrounding HCQ as their reason for termination, about the same number as those citing participant competition. This scenario speaks to a concerning lack of responsiveness by the trialist community to emerging evidence. While there was arguably reason to investigate HCQ in early 2020, the determination by multiple large scale studies of lack of efficacy as of the third quarter of 2020 should have led to definitive determinations about its usefulness, the end of evidence-based uncertainty, and the discontinuation of now-unnecessary trials [33]. Determining when to suspend a trial in relation to emerging evidence is methodologically and ethically complex, but it would likely be generally accepted that a suitable standard would include stability of a systematic review and/or multiple RCTs arriving at similar conclusions.

Trialists should be expected to disseminate their results globally as soon as they are adequately vetted, and to promptly update central registries and regulatory bodies with those results. A centralized registry could include as a function, for example, the ability to issue live updates to all investigators, sponsors or regulators who are attached to registered trials whose core topic relates to a study whose status has been updated.

These results also raise questions of methodology and ethics in relation to evolving knowledge. RCTs of hydroxychloroquine have simultaneously used the drug as investigational agent in comparison to standard care or placebo, in direct comparison to one other treatment, or as the standard care arm in trials where the comparator is standard care plus another treatment. It is difficult to understand how, from a purely evidence-based perspective, one treatment can serve all of these functions at the same moment in time – it is either an established treatment or it is not. We presume that this scenario is a reflection of the complex regulatory landscape that developed in the wake of early reports of HCQ's promise; for example, use of HCQ for COVID-19 was permitted by the US Food and Drug Administration under emergency use authorization on March 28, 2020, but that status was subsequently revoked on June 15, 2020. The rapid shifts of this regulatory environment raise important questions about the ethics of trial design in relation to evolving data.

Third, the case of HCQ for COVID-19 demonstrates the weakness of the frequently-cited concept of “equipoise,” which is often used to justify the conduct of RCTs [7]. Equipoise generally refers to the notion that the ethical acceptability of an RCT relates to some determination of uncertainty on the basis of individual physician or physician-community opinion. However, several different definitions have been offered, and no clear operationalization exists. Indeed, a recent study of RCTs for treatments of COVID-19 has documented that equipoise was frequently mentioned in relation to these trials, though its use was not consistently tied to one clear or specific meaning [34]. In addition to highlighting these inconsistencies in its use, we believe the case of HCQ for COVID-19 demonstrates the limitations of Freedman's popular version of equipoise, namely that it relates to the presence of uncertainty among a community of experts [35]. Importantly, there was no empirical study to demonstrate such uncertainty; therefore any claim about the existence of uncertainty would be hypothetical. Moreover, at the outset of the pandemic, there would have been no evidence-based foundation for physician opinion about HCQ for COVID-19 in that no data were available to render such an opinion informed. Later, physician opinion would at best have served as an indirect measure of a rapidly evolving evidence base. Finally, once multiple RCTs had arrived at a definitive determination of the efficacy of HCQ, physician opinion would have been irrelevant in the face of high quality evidence. As has been advocated by many groups [36,37], we contend that, in general, the determination of the need for first or further RCTs of a treatment for a condition should be based on a systematic review of existing literature – what we have termed evidence-based uncertainty [38] – in contrast to “equipoise.” Indeed, technological innovations spurred by COVID-19 have demonstrated how systematic reviews can characterize a rapidly developing knowledge base in real-time to best inform clinical and trial decision-making [39]. When dealing with a novel treatment for an existing disease or a novel disease, a systematic review will not generate any findings and trial justification should instead be informed by mechanistic reasoning rather than by physician opinion. In these circumstances, physician opinion would be meaningless as physicians would not have evidence off of which to base their opinions and consequently equipoise would not be a suitable justification for the conduct of an RCT.

We propose two improvements to the current system in the hopes of preventing a similar situation. First, we suggest that regulators should base their decisions about the scientific justification of a trial on available data rather than on opinion in the medical community, appeals to equipoise, or to political exigencies. Second, we propose that regulatory bodies’ decisions about whether to approve a prospective trial should not just consider available data, but should also consider what is likely to be known at the time of a proposed study's completion. Trialists and their regulators should compare proposed work to trials already registered to see if there are duplicative studies already in progress and, if so, what the completion dates for those projects might be. If there are substantial similarities to ongoing trials, especially trials expected to release results in the near future, then a given trial's sponsor should be expected to explain why his or her proposed trial is sufficiently different from ongoing work to justify its existence. It is by considering that future state of knowledge, which incorporates registered ongoing trials that will allow regulators to limit waste or duplication

Our study has several limitations. We chose to solely analyze the NIH clinical trials registry, due to its size and widespread use. However, by doing so, our counts are a conservative estimate and the full extent of hydroxychloroquine use in SARS-Cov-2 trials remains unknown. We acknowledge the widely known limitations of such registries [40], but at the present time we have no better system for identifying ongoing and planned studies. In addition, a lag can exist between a trial making a major change to its approach and this change being declared on the registry. We see evidence of this, as several studies which are still officially “active” have since published data and no longer appear to be recruiting patients. Finally, we looked only at trials investigating HCQ, and we recognize that thousands of other trials investigating other treatments for COVID-19 – aside from those assessing vaccines – have been undertaken since early 2020. We suspect that the example of HCQ is sufficiently robust to generate useful insights due to the large number of trials we identified and due to its complex medical, political and regulatory dimensions. We did not analyze individual studies for their quality or risk of bias and have not engaged in the broader debate surrounding research oversight which is itself an important topic brought to light in the pandemic era [41].

Our results support the calls from those colleagues who have encouraged the clinical research community to develop trials that are, to the greatest extent possible, pragmatic, agile, resource-conscious, and definitive [42]. Trials should aim to answer clinically-meaningful questions that will have the broadest possible impact on practice. They should respond to evolving medical knowledge, either by altering treatment allocation ratios or by favoring investigational arms that are supported by emerging evidence. Trials should aim to be large and authoritative, preferably both multi-center and multi-regional, so as to maximize generalizability, limit competition for patient enrolment, and allow for continued recruitment if local disease rates fluctuate. This analysis suggests that RCTs of HCQ for COVID-19 failed to fulfill these objectives in many instances.

Author statement

BD, VY and MS conceptualized the work, performed the investigation and formal analysis, created the methodology, and wrote the original draft. MS was responsible for funding acquisition, resources, and supervision. TR was responsible for data validation and formal analysis. DD, CD, EK, TR were responsible for funding acquisition, review, editing, and data validation. MM and HL were responsible for review, editing and data validation. All authors approved this work.