Predicting outcomes in COVID-19: From internal validation to improving care.

One of the major challenges in treating patients with COVID-19 is predicting the severity of disease [1]. In the context of a healthcare system stretched to capacity, the identification of factors associated with outcomes in COVID-19 is critically important [2]. Initially, COVID-19 was thought to be associated with a cytokine storm [3]. Subsequently, cytokines and particularly IL-6 have attracted much attention as potential outcome biomarkers. However, it has been relatively difficult to consistently link poor clinical outcomes with baseline plasma concentrations of IL-6. A recent report indicates that IL-6 and IL-8 levels in critically ill patients with COVID-19 are considerably lower than in those with septic shock with or without the acute respiratory distress syndrome (ARDS) [4]. Measures of individual cytokines at initial presentation may thus provide limited information on the clinical course of COVID-19. An alternative approach would be to study a composite cytokine profile and its change over the course of disease.

In this issue of EBioMedicine, McElvaney and colleagues [5] propose a new tool to aid in predicting the clinical outcome of patients hospitalized with COVID-19 based on the change observed over 4 days of the cytokine ratio of IL-6 to IL-10. The Dublin-Boston score is a 5-point scale, based on this change observed in 80 patients hospitalized with a confirmed diagnosis of COVID-19. Although baseline IL-6 was weakly associated with clinical outcome, the change in IL-6:IL-10 over time, particularly after 4 days, proved to be a far better predictor of outcome.

In terms of prognostic research, this is a model development and internal validation study for the Dublin-Boston score [6]. An exciting contribution of the study is identifying a new prognostic factor and demonstrating its superiority over IL-6. If the IL-6:IL-10 ratio is confirmed in a broader study population to have significant prognostic value, it can be widely used for prediction models related to COVID-19 and, perhaps, as a biomarker of treatment response.

The predicted outcome of the Dublin-Boston score is a relative difference in clinical status between day 0 and 7, while the prediction is made using information between day 0 and 4. In order to demonstrate that it is potentially useful for clinicians, it is necessary that the predicted outcome not overlap known outcomes. Subsequent studies also need to demonstrate that the use of the model improves clinical decisions over not using the model [7]. The very determination of a “declined” or “improved” clinical status in the study hinged on the ability of the existing health care system to identify a change in status and make the appropriate decision to step up or step down care. Ideally, one should demonstrate that the model is [1] better at making this assessment than usual parameters such as vital signs, mental status, kidney and liver function, and [2] that the clinical application of the model-based tool improves outcomes. It is important to emphasize that the ultimate indicator of a clinical prediction model's worth is its ability to impact care [8]. Consistent with an internal validation study, the excellent performance characteristics of the model in this study should not be used to infer the potential usefulness of the model, only how it performs relative to other models in the same study set.

The authors make the interesting comment that the IL-6:IL-10 ratio predicts outcomes but should not necessarily be used as a therapeutic target. There is an increasing awareness that prediction models should not only be predictive of the outcome, but should use predictors where the causal relationships with the outcome are more fully understood [9]. If the IL-6:IL-10 ratio is not causally responsible for a change in status, then it is possible that unmeasured confounding factors, such as the administration of tocilizumab as highlighted by the authors, can change the observed value in a way that makes model predictions inaccurate. Two questions arise: First, what else has the potential to confound the predictive ability of the IL-6:IL-10 ratio in clinical practice? And second, if the IL-6:IL-10 ratio is not causally related, then what is? The first is the concern of a prediction model researcher who wants to make a useful model while the second is the concern of a basic science researcher who wants to explain the disease and find treatments, but both are fundamentally related.

Finally, as more treatments move to the bedside, the Dublin-Boston score or the IL-6:IL-10 ratio could provide a useful tool to monitor response and support decisions to initiate or change therapies. Subsequent validation studies should include sensitivity analyses for patients on new therapies such as high-dose steroids, as is recently indicated for patients with severe and critical COVID-19 [10].

Overall, the authors make a compelling case for studying composite cytokine profiles as biomarkers for patients with COVID-19. Hopefully, the initial promise of the IL-6:IL-10 ratio evolution will be pursued in subsequent studies to determine its impact on care.