Letter regarding "Association of liver abnormalities with in-hospital mortality in patients with COVID-19".

To the Editor:

With great enthusiasm, we read a breakthrough study demonstrating the association of liver abnormalities with in-hospital mortality in patients with COVID-19. Integrating 10 candidate predictor parameters determined by multivariate regression analyses, Ding and colleagues formulated a novel prognostic nomogram to predict the survival of patients with COVID-19. However, several crucial pitfalls should be taken into account.

In the study of Ding et al., a total of 2,073 patients with COVID-19 were involved. Subsequently, 10 candidate predictor parameters (including age, severe pneumonia, lymphocyte count, PLT, CRP, D-dimer, creatinine, cTnl, AST, and D-Bil) were selected to construct the prognostic model and plot the nomogram. However, is the limited sample size large enough to establish the prediction model which can predict the overall survival probability of patients with COVID-19? As a matter of fact, we are skeptical about the reliability of this nomogram. In the authoritative methodological research published in the journal of BMJ, Riley et al. pointed out that a robust prediction model should be constructed based on the required sample size that is large enough to sufficiently target precious model prediction and minimize model overfitting. Thus, the required sample size of Ding et al. was calculated according to formulas of Riley et al. , Firstly, lnLnull is , and max(R 2 CS) is . Secondly, the conservative value of R2 cs is 0.1068%=0.008. Thirdly, we inputed a key time point to predict the overall survival probability (14 days = 0.038 year), alongside the number of candidate predictor parameters (n = 10), the anticipated mean follow-up (37.81 days = 0.104 year), the mortality rate (0.096), and the conservative value of R2 cs (0.008). Finally, the minimum required sample size of 14-day survival prediction for Ding et al.’s study was calculated in Stata software with the following codes:

pmsampsize, type(s) rsquared(0.008) parameters(10) rate(0.096) timepoint(0.038) meanfup(0.104)

The result is shown in Fig. 1 A, indicating that at least 11,200 patients are required for 14-day survival prediction, corresponding to 1,164.8 deaths and an EPP (events per candidate predictor parameter) of 11.18. In addition, the minimum required sample sizes of 21-day and 28-day survival prediction were also calculated (Fig. 1B and C). Therefore, the minimum required sample size to construct the prognostic nomogram should be 11,200 patients, which is much larger than the sample size (2,073 patients) of Ding et al.’s study.

Fig. 1

The minimum required sample sizes for survival predictions in Ding et al.’s nomogram calculated by the pmsampsize package in Stata software.

(A) 14-day survival prediction. (B) 21-day survival prediction. (C) 28-day survival prediction. The results indicated that at least 11,200 patients are required for 14-day, 21-day, and 28-day survival predictions, corresponding to 1,164.8 deaths and an EPP of 11.18. EPP, events per candidate predictor parameter. (This figure appears in color on the web.)

Moreover, though the calibration curve showed good consistency, the ROC (receiver operating characteristic) curve and DCA (decision curve analysis) curves were eagerly warranted. , The ROC curve can evaluate the discriminative ability of the nomogram, while the DCA curve can evaluate the clinical usefulness of the nomogram. Furthermore, Ding et al.’s nomogram would be more valuable if they applied the disease-specific survival to establish the prognostic nomogram. After all, the overall survival may be impacted by other factors except for COVID-19.

We appreciate Ding et al. for their vital thoughts on the association between liver abnormalities and COVID-19, which has paved a novel way to predict the overall survival probability of patients with COVID-19. However, the minimum required sample size to construct the prognostic nomogram in Ding et al.’s study should be 11,200 patients. The discriminative ability and the clinical usefulness of the nomogram were definitely necessary. Consequently, these above issues may strongly impact the reliability and applicability of Ding et al.’s nomogram.

Financial support

The authors received no financial support to produce this manuscript.

Authors' contributions

G-J.H and M-S.L designed the letter; M-S.L wrote the letter; G-J.H revised the letter.

Conflict of interest

The authors declare no conflicts of interest that pertain to this work.

Please refer to the accompanying ICMJE disclosure forms for further details.