Development and Validation of Clinical Prediction Models to Estimate the Probability of Death in Hospitalized Patients with COVID-19: Insights from a Nationwide Database.

We have read with interest the recent work of Tanboga et al., entitled “Development and Validation of Clinical Prediction Models to Estimate the Probability of Death in Hospitalized Patients with COVID‐19: Insights from a Nationwide Database.” The authors analyzed data from a national database of 60 980 patients with coronavirus disease 2019, hospitalized during the first 3.5 months of the pandemic in Turkey, to develop a model for predicting in‐hospital deaths up to 30 days of hospitalization, and the model was validated across geographic locations and in two subsequent time periods. The final multivariate logistic regression model presented includes age, heart failure, diabetes, detection of pneumonia on computed tomography, and some hematological and serological parameters at baseline (Tab. 3).

To our knowledge, this is the largest cohort study for predicting 30‐day in‐hospital fatality (of 4.0%) among hospitalized patients with at least one positive reverse‐transcription polymerase chain reaction test for severe acute respiratory syndrome coronavirus 2. The findings affirm those obtained earlier from multicenter cohorts or national registries elsewhere.2, 3, 4, 5 Nonetheless, conclusions are still prone to accuracy and efficiency discussions due to insufficient data on patient‐/disease‐related characteristics obtained through electronic patient records and the modeling strategy (i.e., backward elimination method, lacking clinical endorsement for proven predictors of prognosis, such as steroid use, and/or potential effect modifiers). It is not clear how many variables included in models had missing values (<20%) and whether/how the multiple imputations used could have biased the final risk estimates and relevant confidence intervals. The predictive performance of the models and validation across time and location is reported to be high, in the absence of comprehensive indices of health status and disease severity at baseline, the exact timing of various lab test and/or the types of treatment modalities used. It is noteworthy, however, that R 2 values range between 0.324 and 0.532 upon validation (Tab. 4), signaling a need for further investigation of other covariates to lead clinicians properly for minimizing fatality and for studying potential interactions with environmental factors. Baseline measures are important in predicting mortality to prioritize high risk patients, yet, fatality‐related models should emphasize the role of clinical‐based decisions and choices, as well. Feasibility and timeliness of proper care is also an important factor to be analyzed: It is remarkable that more than one‐fourth of the patients died in the first 24 h of admission to the intensive care unit (ICU). This might be due to the severity of cases per se or the delay in transfer to ICU due to high patient load.

We would disagree with the authors' rationale for using logistic regression model “because data on outcome were almost complete, thus, censoring could not provide further benefit to the model.” It is not clear how many of the patients fully recovered, and what percentages of those censored at Day 30 died afterward. Together with the variation in the length of hospitalization (Figures S1–S2) and the timing of laboratory tests, the absolute values of the risks estimated per predictor might have been biased upon pooling patients recovered and those with ongoing disease/hospital stay in the same category of “alive” in logistic regression models. We would rather use survival models in studying the associations between various laboratory tests and death, to adjust for the exact time between initiation of the risk (such as high creatinine values) and outcome by Day 30.

From a clinical point of view and practical benefits, treating laboratory variables as continuous measures, with restricted cubic spline transformations (four knots) to capture nonlinear associations, might not be the best approach.1, 6 There are critical thresholds for several tests commonly used in clinics for patient triage or estimation of prognosis; such thresholds or the values obtained from earlier work could be used, when appropriate. If the authors revealed that the effect of potential predictors on death was not linear and the observed diffraction values did not match those in clinical practice (Fig. 1), we would rather use dummy coding for such variables. As presented (Tab. 3), the simple model implies a 4.3 fold increase in the risk odds ratio of 30‐day in‐hospital deaths for 1‐year of an increasing age, adjusting for other risk factors; it is not in line with our clinical experience of an increasing slope after 50 years. We would also criticize the final model for excluding sex. The authors might have found it nonsignificant, yet, excluding sex from the model jeopardize the comparability of the findings across studies.2, 3, 4, 7

Despite intrinsic limitations of observational studies using registries, such studies are valuable for minimizing analysis time, whilst using large samples with real‐life experiences. Modeling in the pandemic should seek beyond statistical performance: models should be comparable to discriminate local/national differences (if any), be explanatory yet parsimonious so that it entails limited data collection, and to integrate clinically meaningful and modifiable factors to support clinicians in their decisions to prevent death, beyond anticipation. Multidisciplinary efforts and support from academia would improve the quality and effective usage of the models reached.