Hydroxychloroquine for COVID-19: Balancing contrasting claims.

The place of hydroxychloroquine in the prevention and treatment of COVID-19 has been controversial, with contrasting claims and high-emotions. It was initially proposed as treatment or prophylaxis for COVID-19 based upon data reporting that chloroquine and hydroxychloroquine inhibited SARS-CoV-2 virus replication in cell culture systems. Interest accelerated rapidly as early reports from observational studies of clinical efficacy in COVID-19 patients emerged and hydroxychloroquine soon began to appear in clinical treatment guidelines. Despite the lack of reliable evidence of efficacy or safety in clinical trials of patients with COVID-19, hydroxychloroquine was approved for emergency use outside of clinical trials by several national drug regulators.

Although it is perhaps a perfectly natural desire to use observational data to draw inferences about the effects of treatment, doing so is fraught with danger [1]. Most treatment effects are only moderate in size (perhaps reducing risk only by about one quarter). Consequently, any clinical study seeking to estimate such an effect must ensure that biases and random errors inherent in their design are both substantially smaller than the treatment effect to be measured. The avoidance of moderate random errors can be achieved by obtaining a large enough sample size, but the only way to guarantee the avoidance of moderate systematic errors (biases) is to randomize. Non-randomized observational studies – irrespective of how they are analyzed – cannot be guaranteed to eliminate moderate biases arising from the failure to know with certainty why some patients receive a drug and others do not [2]. As a result, large apparent effects in observational studies can arise due to unmeasured or residual confounding alone [3].

It is with this in mind that the findings from the CORIST Collaboration reported in this issue must be considered [4]. In their analyses, use of hydroxychloroquine among 3451 hospitalized patients with COVID-19 was associated with 30% lower 35-day mortality, with a 95% confidence interval around that estimate ranging from 17% to 41%. Because of the observational study design, the authors took several measures to try to control for potential sources of bias, including adjustment for covariates, multiple imputation of missing data, adjustment for hospital clustering and analyses using inverse probability for treatment weighting by propensity scores. Sensitivity analyses supported their main findings.

On the face of it, these results might seem to provide persuasive evidence that hydroxychloroquine reduces mortality in hospitalized COVID-19 patients (rather than merely being associated with a reduction in mortality), but how does this result compare with the randomized evidence? Around the same time the CORIST Collaboration's article was submitted, results from the RECOVERY randomized trial of 4716 hospitalized COVID-19 patients randomized to hydroxychloroquine vs usual care were announced. In RECOVERY, randomization to hydroxychloroquine resulted in a 9% increase in 28-day mortality (with a 95% confidence interval around that estimate ranging from a 3% reduction to a 23% increase) [5]. The difference between these two study results is substantial, with less than a 1 in 20,000 chance that such a difference could have arisen solely to the play of chance, so how does one interpret this?

Fortunately, the CORIST Collaboration's own interpretation was appropriately measured, stating in the Discussion that ‘…the observational design of our study does not allow to fully excluding the possibility of residual confounders’. Given the result from RECOVERY – which was subsequently supported by a similar result from the WHO's SOLIDARITY trial [6] – it seems likely that residual bias probably does provide the explanation. That is, despite the efforts made by the authors to control for bias, it was insufficient to fully remove the fact that those who received hydroxychloroquine were (before they were given it) systematically healthier than those who did not receive it. As an example, one potential unmeasured confounder in the CORIST study is the presence of dementia. In a large cohort of patients hospitalised with COVID-19, the presence of dementia was associated with an increased risk of in-hospital mortality (hazard ratio for death of 1.49, 95% confidence interval 1.28 to 1.52) [7]. If patients with dementia are less likely to be given hydroxychloroquine than patients without dementia, which is plausible, there will an imbalance between groups in this risk factor for death.

In the absence of robust evidence of safety and efficacy, the use of unproven drugs such as hydroxychloroquine outside of clinical trials has a number of potential adverse consequences [8]. First, there is potential for direct harm to patients. A risk-benefit balance in favor of the drug should not be assumed from observational data alone, and both the RECOVERY and SOLIDARITY trials have reported a possibility of excess mortality risk associated with the receipt of hydroxychloroquine (albeit at higher doses than reported by the CORIST Collaboration). Second, emergency use authorization can jeopardize enrolment into randomized controlled trials thereby slowing progress towards definitive evidence. In the United States, emergency use authorization of both hydroxychloroquine and convalescent plasma has resulted in tens of thousands of patients being given these drugs without parallel accumulation of randomized evidence. Third, there is a risk that emergency use becomes embedded in routine practice, as has happened with the use of neuraminidase inhibitors in patients hospitalized with influenza despite the lack of evidence from randomized trials of the efficacy of these drugs in complicated influenza [8]. Once an unproven drug becomes standard of care, all future drug development is hampered by the need to compare new interventions against an unknown quantity, necessitating the demonstration of superiority since equivalence to an unknown is uninformative.

Analyses such as those reported by the CORIST Collaboration can of course be used to generate new hypotheses about the effects of treatment in different circumstances (including the possibility that a lower dose of hydroxychloroquine or treatment of patients earlier in their disease might have a different effect to those observed in RECOVERY and SOLIDARITY) but such findings should not be used as evidence that such a hypothesis is true. For that, as the CORIST Collaborators acknowledge, large-scale randomized trials are needed.

Disclosures

Peter W. Horby is co-Chief Investigator of the RECOVERY trial. Jonathan R. Emberson is a statistician for the RECOVERY trial.