Response to "Overlooked Shortcomings of Observational Studies of Interventions in Coronavirus Disease 2019: An Illustrated Review for the Clinician" by Tleyjeh et al.

Dear Editor,

We read with great interest the recent publication by Tleyjeh et al. regarding the overlooked shortcomings of observational studies of interventions in coronavirus disease 2019 (COVID-19) [1]. Similarly, the issue of bias in observational studies assessing drug effectiveness in hospitalized patients with COVID-19 was described in our methodological review [2]. In our review, we focused on the studies published in 4 leading clinical journals and examined the presence of 3 main types of biases, namely immortal time bias, type-dependent confounding bias, and competing risk bias. In the majority of the reviewed studies, the primary outcome was in-hospital death, and discharge alive was considered a competing event by these studies. Overall, all of the 11 assessed studies were prone to the competing risk bias. Among them, only 1 study addressed the competing risk bias. In studies, conventional methods such as the naïve Kaplan-Meier approach and a standard Cox regression model were applied. In using the Kaplan-Meier methodology, the competing event (ie, discharge alive) was treated as a censored observation, potentially leading to biased estimates of the primary event probabilities. However, this is not a meaningful model assumption, as the recovered patients were not representative of those who were still hospitalized in terms of their risk of dying. Furthermore, the studies applied the standard Cox regression analysis, which is incomplete in the presence of competing events [2]. With this, we want to emphasize the same conclusions derived by Tleyjeh et al. on the importance of competing risk analysis in studies evaluating intervention in COVID-19.

In addition, Tleyjeh et al. provided recommendations on the application of the Fine-Gray subdistribution hazard model for the competing risk analysis. We agree that in the presence of competing risks the subdistribution hazard model allows for estimating the effect of time-invariant covariates on the cumulative incidence of the outcome; however, it may no longer be suitable with internal time-dependent covariates, such as treatment exposures or biomarkers. The subdistribution hazard model requires that values of internal time-dependent covariates be known for the entire follow-up time for patients who experienced a competing event. However, in competing risk settings, the information on time-dependent covariates is mostly unavailable. Thus, the Fine-Gray approach often does not allow for making inferences about the association of an internal time-dependent covariate with the cumulative incidence function [3]. The subdistribution approach can produce highly biased hazard ratio estimands in the assessment of the time-dependent covariate, for example, for time-varying treatment; for example, a simulation showed that this model produced strong effects in settings without any true treatment effect [4]. Thus, when studying potential treatment effects on clinical outcomes in COVID-19 patients, more sophisticated models for time-dependent data are required [5].