Comment on Arshad et al.: Treatment with Hydroxychloroquine, Azithromycin, and Combination in Patients Hospitalized with COVID-19.

We read, with great interest, the results published by Arshad and colleagues concerning the potential association between treatment with hydroxychloroquine with or without azithromycin and in-hospital mortality in patients with COVID-19 (Arshad et al., 2020). The reported treatment benefit contradicts that reported elsewhere, including a recent study at US Veterans Affairs Hospitals that showed an almost perfectly inverse risk of mortality (HR = 2.61; 95% CI, 1.10–6.10) (Magagnoli et al., 2020). Although Lee et al. provided an excellent commentary on the potential for immortal time bias and selection bias in this work (Lee et al., 2020), the following design and analysis flaws further threaten the validity of the reported findings.

First, corticosteroid use was common in patients who received hydroxychloroquine with or without azithromycin, 79%, and 74%, respectively. It is unclear when in the course of care, adjunctive therapy, including steroids and tocilizumab, was initiated. Without disclosing the treatment protocol at each of the six centers, all we can conclude is that steroid use was not consistent at baseline and, therefore, should have been treated as a time-dependent covariate (Vatcheva et al., 2016). This may indicate that an initial clinical decline in the hydroxychloroquine arms was masked by subsequent corticosteroid initiation.

The authors’ efforts to control confounding create additional confusion. Reportedly, 25% of mSOFA scores were missing and “no imputations…[were] made for missing data…” Yet the Cox model was adjusted for “clinical disease severity (mSOFA, O2 saturation).” Table 2 states that all 2,541 cases were included yet shows no effect for mSOFA score, indicating that either all cases were included or the model was not adjusted for severity, as discussed in the text. Dichotomizing age is also concerning, given the established association between COVID-19 mortality and age. Dichotomizing age robs information from the model and is a significant threat to validity.

Finally, in the propensity scored (PS) model, the authors redefined exposure as hydroxychloroquine, or nothing; the authors did not include azithromycin as a covariate in the PS generating model. With so few matched pairs included in the model (n = 190), the omission of this potential confounder, especially one the Cox model showed to be associated with the outcome, should not be taken lightly. Furthermore, we are left to question why exposure was redefined, given the availability of established methods to compare multiple treatment groups using inverse probability of treatment weighting (Kilpatrick et al., 2013). In light of these significant threats to validity, we urge clinicians who treat patients with COVID-19 to interpret and apply the contradictory findings of this study cautiously.

Conflict of interest statement

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding source

No funding source was used for this commentary.

Ethical approval

No ethical approval was necessary for this commentary.