A Rush to Judgment? Rapid Reporting and Dissemination of Results and Its Consequences Regarding the Use of Hydroxychloroquine for COVID-19.

The coronavirus disease 2019 (COVID-19) pandemic has placed the scientific and research communities under extraordinary pressure, to which they have responded with exceptional vigor and speed. This desire to quickly find safe and effective treatments may also lead to relaxed standards of data generation and interpretation, which may have undesirable downstream effects. The recent publication of a study evaluating hydroxychloroquine (HCQ) in COVID-19 is a useful test case, highlighting the challenges of conducting research during a pandemic.

A scientific rationale existed for investigating HCQ in COVID-19. Preclinical data suggested that the antimalarials HCQ and chloroquine have in vitro antiviral activity against severe acute respiratory syndrome–coronavirus 2 (SARS–CoV-2) (1–3). Antimalarials are also inexpensive, are widely available, and have an acceptable short-term safety profile. In a nonrandomized study, Gautret and colleagues (4) reported a higher frequency of SARS–CoV-2 clearance from the nasopharynx after 6 days of treatment with HCQ, plus azithromycin (AZM) if deemed necessary, versus an untreated control group (14 of 20 patients [70%] vs. 2 of 16 patients [13%]; P < 0.001). Given the urgency of the situation, some limitations of this study may be acceptable, including the small sample size, use of an unvalidated surrogate end point, and lack of randomization or blinding. However, other methodological flaws also noted by others (5) may affect the validity of the findings, even in the current setting, where an efficacious treatment is desperately needed.

First, potentially substantial confounders may explain the findings. The HCQ + AZM treatment group was recruited from a single center. Instead of excluding patients who declined treatment, the researchers assigned them to the control group. The remainder of the control group was recruited from other centers that could not contribute to the treatment group. This introduces the potential for baseline confounding and different treatment regimens at different institutions. In addition, patients in the HCQ + AZM group had lower viral loads at the time of treatment initiation compared with the control and HCQ groups, and so may have been at a later phase of infection. All patients who received HCQ + AZM had a SARS–CoV-2 baseline cycle threshold (Ct) greater than 22 on polymerase chain reaction (PCR). Of the 5 patients receiving HCQ who had a baseline Ct of 22 or less on PCR (that is, higher viral burden), 4 still had detectable virus at day 6. Of the 9 patients with a baseline Ct greater than 22 on PCR, only 2 had detectable virus at day 6. Thus, another explanation is that the baseline viral load, not therapy with HCQ + AZM, affects viral load at day 6.

This problem may have been further exacerbated by issues with data measurement and imputation. Sites other than Marseille did not perform daily PCR testing, creating gaps in PCR data for control group patients that were later imputed with values from other days. Consequently, 75% (12 of 16 patients) of the control group lacked at least 1 PCR result, including 44% (7 of 16 patients) who were not tested on at least 4 of the 7 possible days. In the HCQ + AZM group, only 20% (4 of 20 patients) were missing at least 1 PCR result and none were missing more than 2. In the control group, 38% of the PCR data were imputed versus only 5% in the treatment group.

Second, the handling of patients who were lost to follow-up also raises serious questions about scientific validity. Only 20 of 26 patients in the treatment group were included in the analysis despite meeting baseline eligibility criteria. Six patients were excluded because day 6 PCR data were missing, owing to early treatment cessation due to nausea (n = 1), hospital discharge (n = 1), intensive care unit transfer (n = 3), and death (n = 1). Therefore, patients who had the most serious and clinically relevant outcomes, including intensive care unit transfer and death, were excluded from analysis. These patients had treatment failure and should have been analyzed as such. We strongly agree with others that adequate follow-up of patients with relevant outcomes should be attempted (6).

Despite the study's substantial limitations, a simplification and probable overinterpretation of these findings was rapidly disseminated by the lay press and amplified on social media, ultimately endorsed by many government and institutional leaders. Public interest in HCQ rapidly grew (Figure). The study's findings were extrapolated to include the use of HCQ to prevent COVID-19 infection or as postexposure prophylaxis, indications for which there are currently no direct supporting data. Despite promising in vitro data for influenza (7, 8), HCQ failed to prevent infection in a randomized, placebo-controlled, double-blind trial (9). Efforts to understand the clinical efficacy of HCQ as postexposure prophylaxis are under way (https://clinicaltrials.gov/ct2/show/NCT04308668), which should yield important insight into this issue.

Figure.

Global Google Trends search patterns for “hydroxychloroquine,” “chloroquine,” and “hydroxychloroquine shortage,” 16 to 22 March 2020.

The spike on 17 March corresponded with the publication of Gautret and colleagues' report (4). The second spike on 20 March followed the U.S. presidential press conference in which hydroxychloroquine was described as a treatment of coronavirus disease 2019.

A major consequence has been an inadequate supply of HCQ for patients in whom efficacy is established. Hydroxychloroquine is an essential treatment of rheumatoid arthritis and of systemic lupus erythematosus, reducing flares and preventing organ damage in the latter disease (10). Pharmacies have reported shortages of antimalarials (www.washingtonpost.com/business/2020/03/20/hospitals-doctors-are-wiping-out-supplies-an-unproven-coronavirus-treatment), and patients with rheumatic diseases have had difficulty obtaining prescription refills. Several major medical organizations released a joint statement regarding the HCQ shortage (www.lupus.org/s3fs-public/pdf/Joint-Statement-on-HCQ-LFA-ACR-AADA-AF.pdf), warning of possible dire consequences for patients with rheumatic diseases. Hydroxychloroquine shortages could place these patients at risk for severe and even life-threatening flares; some may require hospitalization when hospitals are already at capacity. Until reliable evidence is generated and adequate supply chains have been put in place, rational use of HCQ in patients with COVID-19 must be emphasized, such as use in investigational studies.

In critical situations, large randomized controlled trials are not always feasible or ethical, and critically ill patients may need to be treated empirically during times of uncertainty. However, it is our responsibility as clinicians, researchers, and patient partners to promote proper and rigorous interpretation of results, particularly in our interactions with the nonscientific community. We must consider the societal implications of published work in these unprecedented times.

There is enough rationale to justify the continued investigation of the efficacy and safety of HCQ in hospitalized patients with COVID-19. It is critical to reiterate that although viral clearance is important, clinical outcomes are much more relevant to patients. There currently are no data to recommend the use of HCQ as prophylaxis for COVID-19, although we eagerly await data from trials under way. Thus, we discourage its off-label use until justified and supply is bolstered. The HCQ shortage not only will limit availability to patients with COVID-19 if efficacy is truly established but also represents a real risk to patients with rheumatic diseases who depend on HCQ for their survival.