IL-6-based mortality risk model for hospitalized patients with COVID-19.

BACKGROUND: Coronavirus disease 2019 (COVID-19) has rapidly become a global pandemic. Because the severity of the disease is highly variable, predictive models to stratify patients according to their mortality risk are needed.
OBJECTIVE: Our aim was to develop a model able to predict the risk of fatal outcome in patients with COVID-19 that could be used easily at the time of patients' arrival at the hospital.
METHODS: We constructed a prospective cohort with 611 adult patients in whom COVID-19 was diagnosed between March 10 and April 12, 2020, in a tertiary hospital in Madrid, Spain. The analysis included 501 patients who had been discharged or had died by April 20, 2020. The capacity of several biomarkers, measured at the beginning of hospitalization, to predict mortality was assessed individually. Those biomarkers that independently contributed to improve mortality prediction were included in a multivariable risk model.
RESULTS: High IL-6 level, C-reactive protein level, lactate dehydrogenase (LDH) level, ferritin level, d-dimer level, neutrophil count, and neutrophil-to-lymphocyte ratio were all predictive of mortality (area under the curve >0.70), as were low albumin level, lymphocyte count, monocyte count, and ratio of peripheral blood oxygen saturation to fraction of inspired oxygen (SpO2/FiO2). A multivariable mortality risk model including the SpO2/FiO2 ratio, neutrophil-to-lymphocyte ratio, LDH level, IL-6 level, and age was developed and showed high accuracy for the prediction of fatal outcome (area under the curve 0.94). The optimal cutoff reliably classified patients (including patients with no initial respiratory distress) as survivors and nonsurvivors with 0.88 sensitivity and 0.89 specificity.
CONCLUSION: This mortality risk model allows early risk stratification of hospitalized patients with COVID-19 before the appearance of obvious signs of clinical deterioration, and it can be used as a tool to guide clinical decision making.

The outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection started in December 2019 in China and has since spread globally, reaching more than 7 million confirmed cases and more than 400,000 reported deaths to date. SARS-CoV-2 infection causes Coronavirus disease 2019 (COVID-19), and its severity ranges from asymptomatic to acute respiratory distress syndrome (ARDS), multiorgan failure, and eventually death.

The overall rate of mortality due to COVID-19 has been estimated at 1% to 3%, whereas the reported mortality rate in hospitalized patients has varied from 6% to 34%,4, 5, 6 and it can exceed 50% in patients in intensive care units (ICUs). Given these differences in disease severity and the potentially high in-hospital mortality, there is a need for early predictive biomarkers able to stratify patients according to their likelihood of a fatal outcome. A few reports have described the clinical characteristics of hospitalized patients with COVID-19 and have proposed risk factors for mortality. , 8, 9, 10 These factors include older age; high sequential organ failure assessment (SOFA) score; high neutrophil count; high C-reactive protein (CRP), d-dimer, or lactate dehydrogenase (LDH) or IL-6 level; and low lymphocyte count and platelet count or albumin level.

IL-6 may play a relevant role in the pathophysiology of patients with severe COVID-19. Blocking of the IL-6/IL-6R signaling pathway with tocilizumab among other drugs is currently being studied in clinical trials as a potential treatment for such patients. In this study, we analyzed the individual mortality prediction capacity of early, 1-time, readily available laboratory tests, including tests to determine IL-6 level, together with peripheral blood oxygen saturation–to–fraction of inspired oxygen (SpO2/FiO2) ratio, and we developed a multivariable model to predict fatal outcome in hospitalized patients with COVID-19.

Methods

Study design

This is a single-center prospective cohort study performed at Hospital Universitario 12 de Octubre, a large tertiary hospital in Madrid (Spain). A total of 611 hospitalized adult patients in whom COVID-19 was diagnosed between March 10 and April 12, 2020, and whose IL-6 level had been measured were included in the study. SARS-CoV-2 real-time RT-PCR assay of nasopharyngeal or oropharyngeal swabs or sputum samples was performed to diagnose COVID-19. According to the local protocols during the peak of the pandemic, a COVID-19 diagnosis was assumed in 28.9% of patients with suggestive clinical and radiologic presentation and compatible epidemiologic history (Table I ). Two patients with chronic lymphocytic leukemia and a lymphocyte count higher than 70,000 cells/μL were excluded. Only patients who had been discharged or had died by April 20, 2020, were included in the final analysis (N = 501) (Fig 1 ). The institutional clinical research ethics committee approved the study protocol (reference no. 20/167).

Table I

Demographic and clinical characteristics of the cohort

Characteristic	Overall cohort (N = 501)	Survivors (n = 465 [92.8%])	Nonsurvivors (n = 36 [7.2%])	P value
Age (y), median (IQR)	52 (44-60)	52 (44-58)	65 (57-72)	<.0001
Gender, male, no. (%)	317 (63.3%)	292 (62.9%)	25 (69.4%)	.54
Ethnicity, no. (%)				.035
White	272 (54.3%)	241 (52.5%)	27 (75.0%)
Hispanic	213 (42.5%)	204 (44.0%)	9 (25.0%)
Other	16 (3.2%)	16 (3.5%)	0
Previous comorbidities, no. (%)
Hypertension	139 (27.7%)	117 (25.2%)	22 (61.1%)	<.0001
Diabetes mellitus	56 (11.2%)	49 (10.5%)	7 (19.4%)	.11
Dyslipidemia	125 (25.0%)	109 (23.4%)	16 (44.4%)	.008
Thromboembolic event	23 (4.6%)	20 (4.3%)	3 (8.3%)	.28
Obesity	41 (8.2%)	31 (6.7%)	10 (27.8%)	.0002
Chronic pneumopathy∗	54 (10.8%)	47 (10.1%)	7 (19.4%)	.09
Neoplasia	19 (3.8%)	16 (3.4%)	3 (8.3%)	.14
Smoking	91 (18.2%)	80 (17.2%)	11 (30.6%)	.07
SARS-CoV-2 RT-PCR result, positive, no. (%)†	356 (71.1%)	327 (70.3%)	29 (80.6%)	.25
ARDS classification on test day, no. (%)‡
None	281 (56.1%)	277 (59.6%)	4 (11.1%)	<.0001
Mild	113 (22.6%)	109 (23.4%)	4 (11.1%)	.10
Moderate	36 (7.2%)	32 (6.9%)	4 (11.1%)	.31
Severe	67 (13.4%)	43 (9.3%)	24 (66.7%)	<.0001
COVID-19 treatment on test day, no. (%)
Hydroxychloroquine	442 (88.2%)	413 (88.8%)	29 (80.6%)	.17
Azithromycin	387 (77.2%)	362 (77.9%)	25 (69.4%)	.30
IFN-β	32 (6.4%)	26 (5.6%)	6 (16.7%)	.021
Lopinavir/ritonavir	184 (36.7%)	167 (35.9%)	17 (47.2%)	.21
Corticosteroids	123 (24.6%)	106 (22.8%)	17 (47.2%)	.002
Time from onset of illness to hospital admission (d), median (IQR)	8 (6-10)	8 (6-11)	7 (5-8)	.032
Time from hospital admission to laboratory measurements (d), median (IQR)	2 (1-4)	2 (1-4)	2 (1-4)	.87
Length of hospital stay (d), median (IQR)	8 (6-13)	8 (6-13)	7 (5-13)	.62
ICU admission, no. (%)	42 (8.4%)	28 (6.0%)	14 (38.9%)	<.0001

IQR, Interquartile range.

Included pulmonary fibrosis, chronic obstructive pulmonary disease, bronchiectasis, and asthma.

Real-time RT-PCR.

ARD syndrome.

Fig 1

Flowchart of patients included in the study.

Data collection

Clinical data (including demographics and major comorbidities; hospital admission, discharge, and death dates; time from symptom onset to hospitalization; length of hospital stay; and ICU requirement) were collected from the electronic medical records into the study database. Oxygen saturation and COVID-19 treatment were recorded on the day of the blood test. Respiratory function was assessed by measurement of SpO2/FiO2 ratio. The SpO2/FiO2 ratio shows a good correlation with the partial pressure of arterial oxygen (PaO2)/Fio 2 ratio (SpO2/FiO2 = 64 + 0.84 × [PaO2/FiO2]), and unlike the latter, it was available for all the patients. ARDS was classified according to the Berlin criteria, according to which an SpO2/FiO2 ratio higher than 315 indicates no ARDS, an SpO2/FiO2 ratio between 315 and 235 indicates mild ARDS, an SpO2/FiO2 ratio between 148 and 235 indicates moderate ARDS, and an SpO2/FiO2 ratio less than 148 indicates severe ARDS.

Laboratory measurements

IL-6 level was measured with the BD CBA Human IL-6 Flex Set (BD Biosciences, San Diego, Calif) by using a BD Canto II flow cytometer. The results were analyzed with FCAP Array software, version 3.0 (BD Biosciences). IL-6 level measurements following tocilizumab therapy were not included in the study. Other laboratory parameters such as CRP level, albumin level, alanine transaminase (ALT) level, aspartate transaminase (AST) level, LDH level, ferritin level, d-dimer level, fibrinogen level, platelet count, neutrophil count, lymphocyte count, and monocyte count were measured as part of the standard of care. All laboratory measurements were performed on the same day. Neutrophil-to-lymphocyte (N/L) ratio was calculated for every patient. High-sensitivity cardiac troponin level was not included in the analysis given the low number of patients with that measurement.

Statistical analysis

Continuous numeric data have been represented as medians and interquartile ranges and were compared by using the Mann-Whitney U test. Categoric variables have been represented as numbers and percentages and were compared by using the chi-square test or the Fisher exact test when appropriate. Box plots have been represented in Tukey style. Correlation was evaluated with linear regression and the Spearman rank coefficient. Univariate logistic regression was performed for each variable to test association with mortality. We evaluated each variable as a potential biomarker by using receiver operating characteristic (ROC) curves.

The Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) guidelines for development of multivariate prediction models were followed. All the variables with a significant association with fatal outcome, as tested by univariate logistic regression and an area under the ROC curve (AUROCC) value higher than 0.70 were used in the backward stepwise regression. Nonsignificant variables were excluded thereafter. Random forest analysis (with 500 trees) was then used to evaluate the importance of IL-6 level, LDH level, N/L ratio, SpO2/FiO2 ratio, and age to the model, after which 5 sequential logistic regression models for mortality were performed. The discriminative capacity of the 5 models was estimated by using the AUROCC. No imputation was performed; for every model, only patients with all the available data were included in the logistic regression. Model coefficients are shown in Table E1 (available in this article’s Online Repository at www.jacionline.org). The Hosmer-Lemeshow test was used to assess the models' goodness of fit. Logistic regression models were transformed to a 0-1 scale to facilitate their interpretation. The model including 5 variables was selected as our final mortality predictive model. The final model can be calculated with the following formula: score = 1/(1 + EXP [–(–7.6991 – 0.0076 × SpO2/FiO2 ratio + 0.0547 × N/L ratio + 0.0046 × LDH + 0.0043 × IL6 + 0.0682 × age)]). The variance inflation factor was lower than 1.2 for the 5 variables in the model, showing no collinearity. Internal validation of the selected model was performed by bootstrapping (1000 replications) of the 5 variables. The prediction model was calculated on each bootstrap sample and tested on the original sample. Optimism and optimism-adjusted discrimination were calculated.

Table E1

β-Coefficients of sequential regression models for mortality prediction in COVID-19

Model	Intercept	SpO2/FiO2 ratio	N/L ratio	LDH level	IL-6 level	Age
1	0.8415	–0.0136	—	—	—	—
2	–0.3353	–0.0121	0.0660	—	—	—
3	–2.2212	–0.0103	0.0604	0.0034	—	—
4	–3.2413	–0.0091	0.0675	0.0038	0.0046	—
5	–7.6991	–0.0076	0.0547	0.0046	0.0043	0.0682

The Youden index was used for cutoff selection. Sensitivity, specificity, negative predictive value, and positive predictive value (PPV) were calculated for each individual variable and model cutoff. Time-to-event curves were plotted by the Kaplan-Meier method, and differences were compared with the log-rank test to analyze the suitability of the score for stratification across risk categories. Throughout the analysis, only patients with available data were compared, and the number of patients is specified in the figures and tables. A P value less than .05 was considered statistically significant. Data sets can be made available subsequent to formal request to the corresponding author. All the analyses were performed with R software, version 3.6.1.

Results

The final patient cohort included 501 patients with COVID-19 (Fig 1); their demographic and clinical characteristics are described in Table I. In all, 36 patients (7.2%) died during follow-up and 42 patients (8.4%) required ICU admission.

The median time from arrival at the hospital to laboratory testing was 2 days (interquartile range, 1-4 days) (Table I). Neither the time from hospital admission to laboratory testing nor the time from onset of illness to hospital admission differed between survivors and nonsurvivors (2 days vs 2 days [not significant] and 8 days vs 7 days [not significant], respectively). The laboratory test results had minimal or no correlation with the time from onset of illness (see Fig E1 in this article’s Online Repository at www.jacionline.org). However, all of the recorded variables except for AST level and fibrinogen level differed between survivors and nonsurvivors (Table II ). IL-6 level, CRP level, LDH level, ferritin level, d-dimer level, neutrophil count, and N/L ratio were significantly increased in deceased patients compared with in discharged patients, whereas albumin level, ALT level, platelet count, monocyte count, lymphocyte count, and SpO2/FiO2 ratio were decreased. Univariate logistic regression assessed the ability of each individual variable to predict mortality; the analysis showed that high IL-6 level, CRP level, LDH level, ferritin level, neutrophil count, and N/L ratio were risk factors for mortality due to COVID-19, whereas low albumin level, ALT level, platelet count, monocyte count, lymphocyte count, and SpO2/FiO2 ratio were also associated with higher risk of death (see the odds ratio in Table II). d-Dimer level did not reach statistical significance as a risk factor.

Fig E1

Laboratory values and SpO2/FiO2 ratio did not correlate with the time from onset of illness to the measurement. Most variables did not correlate, and for those that did, the impact of days from onset of illness was minimal (very low R2). This implied that the values obtained depended not on the number of days with symptoms until the measurement but rather on the severity of each particular patient.

Table II

Variables measured in survivors and nonsurvivors and their association with mortality

Variables	Survivors (n = 465)			Nonsurvivors (n = 36)			P value	Univariate logistic regression
	Median	IQR	n	Median	IQR	n		OR∗	95% CI	P value
IL-6 level (pg/mL)	17.0	5.0-44	421	86.0	20.0-225.0	33	<.0001	1.008	1.005-1.012	<.0001
CRP level (mg/dL)	7.68	3.31-14.93	464	19.4	13.0-26.5	36	<.0001	1.105	1.069-1.145	<.0001
Albumin level (g/dL)	3.7	3.4-4.0	459	3.3	2.9-3.5	36	<.0001	0.071	0.028-0.168	<.0001
ALT level (U/L)	43.0	28.0-72.0	463	24.5	16.5-39.5	36	.0002	0.984	0.970-0.996	.016
AST level (U/L)	38.0	26.0-58.0	461	42	30.0-64.8	36	.22	1.001	0.991-1.008	.86
LDH level (U/L)	344.0	270.8-443.0	456	538.0	384.8-691.8	36	<.0001	1.007	1.005-1.009	<.0001
Ferritin level (ng/mL)	984.0	449.5-1801.2	378	2345.0	1502.0-3388.0	23	.0001	1.000	1.000-1.000	.0009
d-Dimer level (ng/mL)	550.0	354.8-935.0	304	1514.0	614.5-3367.2	26	<.0001	1.000	0.999-1.000	.052
Fibrinogen level (mg/dL)	811.0	697.0-923.0	331	845.0	776.0-960.5	27	.12	1.002	0.999-1.004	.08
Platelet count (× 103 cells/μL)	277.0	211.5-372.0	463	232.5	172.8-300.8	36	.011	0.996	0.992-0.999	.017
Monocyte count (× 103 cells/μL)	0.5	0.4-0.7	463	0.4	0.2-0.5	36	.002	0.084	0.015-0.398	.003
Neutrophi count (× 103 cells/μL)	4.3	3.0-6.8	463	8.1	6.0-11.4	36	<.0001	1.225	1.135-1.326	<.0001
Lymphocyte count (× 103 cells/μL)	1.0	0.7-1.4	464	0.6	0.3-0.9	36	<.0001	0.076	0.025-0.201	<.0001
N/L ratio	4.2	2.4-8.0	463	16.9	7.4-28.4	36	<.0001	1.097	1.065-1.133	<.0001
SpO2/FiO2 ratio	346.4	269.4-452.4	461	106.7	101.1-212.1	36	<.0001	0.986	0.982-0.990	<.0001

Odd ratio per unit increase.

To evaluate the potential of these variables as predictors of mortality due to COVID-19, an AUROCC analysis was performed (Table III ). Several variables emerged individually as potential biomarkers, namely, IL-6 level (AUROCC = 0.74), CRP level (AUROCC = 0.80), albumin level (AUROCC = 0.81), LDH level (AUROCC = 0.78), ferritin level (AUROCC = 0.74), d-dimer level (AUROCC = 0.75), monocyte count (AUROCC = 0.71), neutrophil count (AUROCC = 0.76), lymphocyte count (AUROCC= 0.79), N/L ratio (AUROCC = 0.83), and SpO2/FiO2 ratio (AUROCC = 0.87). Optimal cutoffs based on Youden index were calculated for these variables to classify patients into groups at high or low risk of a fatal outcome. AUROCC and survival curves for each individual biomarker are shown in Fig E2 (in this article’s Online Repository at www.jacionline.org), except for neutrophil count and lymphocyte count (which were replaced in any further analysis by N/L ratio because of its superiority as a biomarker in this setting). The variable with the highest sensitivity was CRP level (0.97), whereas IL-6 level had the highest specificity (0.89) (Table III). Patients in our cohort with a CRP value below 8.75 mg/dL had less than a 1% probability of dying. PPVs were discrete owing to the relatively low mortality in the cohort. IL-6 level and SpO2/FiO2 ratio had the highest PPVs (0.26 and 0.32 respectively), which meant that having an IL-6 level higher than 86 pg/mL or an SpO2/FiO2 ratio lower than 211 increased the likelihood of dying from 7.2% in the overall cohort to 26% or 32%, respectively. Individually, the most robust biomarkers for predicting risk of mortality in this study were CRP level and SpO2/FiO2 ratio; patients with a CRP level higher than 8.75 mg/dL or an SpO2/FiO2 ratio less than 211 had 20 times more risk of dying than did those with values below or above the threshold.

Table III

Variables analyzed as potential biomarkers for mortality

Variable	ROC curve		Risk factor cutoff characterization					Cutoff univariate logistic regression
	AUC	95% CI	Cutoff	Sensitivity	Specificity	PPV	NPV	Value	OR	95% CI	P value
IL-6 level (pg/mL)	0.74	0.64-0.84	86	0.52	0.89	0.26	0.96	≤86	Ref	—	<.0001
								>86	7.67	3.61-16.30
CRP level (mg/dL)	0.80	0.72-0.89	8.75	0.97	0.53	0.14	0.99	≤8.75	Ref	—	<.0001
								>8.75	19.52	5.85-121.15
Albumin level (g/dL)	0.81	0.75-0.87	3.4	0.74	0.78	0.17	0.97	≤3.4	6.35	3.10-13.80	<.0001
								>3.4	Ref	—
ALT level (U/L)	0.69	0.61-0.77	25	0.80	0.53	0.16	0.96	≤25	4.06	2.03-8.17	.0001
								>25	Ref	—
AST level (U/L)∗	0.57	0.47-0.67	—	—	—	—	—	—	—	—	—
								—	—	—
LDH level (U/L)	0.78	0.69-0.87	424	0.72	0.71	0.16	0.97	≤424	Ref	—	<.0001
								>424	5.70	2.79-12.38
Ferritin level (ng/mL)	0.74	0.62-0.86	1799	0.70	0.75	0.14	0.97	≤1799	Ref	—	.0002
								>1799	5.59	2.35-14.26
d-dimer level (ng/mL)	0.75	0.64-0.87	1386	0.62	0.84	0.23	0.96	≤1386	Ref	—	<.0001
								>1386	7.10	3.10-16.76
Fibrinogen level (mg/dL)∗	0.59	0.47-0.71	—	—	—	—	—	—	—	—	—
								—	—	—
Platelet count (× 103 cells/μL)	0.63	0.54-0.72	245	0.63	0.61	0.11	0.95	≤245	2.61	1.31-5.35	.007
								>245	Ref	—
Monocyte count (× 103 cells/μL)	0.71	0.63-0.78	0.4	0.77	0.67	0.11	0.96	≤0.4	2.63	1.31-5.56	.008
								>0.4	Ref	—
Neutrophil count (× 103 cells/μL)	0.76	0.66-0.85	5.1	0.86	0.62	0.15	0.98	≤5.1	Ref	—	<.0001
								>5.1	8.08	3.53-21.87
Lymphocyte count (× 103 cells/μL)	0.79	0.73-0.85	0.9	0.63	0.83	0.13	0.98	≤0.9	6.37	2.78-17.23	<.0001
								>0.9	Ref	—
N/L ratio	0.83	0.74-0.91	6.5	0.86	0.68	0.17	0.98	≤6.5	Ref	—	<.0001
								>6.5	10.54	4.59-28.55
SpO2/FiO2 ratio	0.87	0.83-0.92	211.11	0.88	0.75	0.32	0.98	≤211.11	21.26	9.86-50.04	<.0001
								>211.11	Ref	—

AUC, Area under the curve; NPV, negative predictive value; Ref, reference.

Variables with an AUC less than 0.60 were not analyzed further.

Fig E2

ROC and Kaplan-Meier analysis of individual variables with an area under the curve (AUC) greater than 0.70 and with a P value less than .05 in univariate logistic regression. Color shades represent the 95% CI. Time is indicated in days.

A multivariate prediction model was developed to improve the predictive power of each individual biomarker. The independent contribution of the top 8 variables to mortality risk, as identified by significant univariate logistic regression (Table II) and ROC curve analysis result greater than 0.70 (Table III) plus age, was assessed by backward stepwise regression. Age was included in the analysis because it had been identified as one of the major risk factors in COVID-19 in previous studies. , Of these 9 variables, only IL-6 level, LDH level, N/L ratio, SpO2/FiO2 level, and age showed a statistically significant individual contribution to the predictive model. Random forest analysis gave the relative levels of relevance of each of the selected variables for mortality risk stratification, which in descending order were as follows: SpO2/FiO2 ratio, N/L ratio, LDH level, IL-6 level, and age (see Fig 3 in this article’s Online Repository at www.jacionline.org). The sequential regression models for mortality prediction in COVID-19 are shown in Table IV , and their β-coefficients are shown in Table E1. Increasing the model's complexity through the sequential addition of biomarkers improved its predictive performance up to an AUROCC of 0.94 (95% CI = 0.89-1.00). This final model, which includes SpO2/FiO2 ratio, N/L ratio, LDH level, IL-6 level, and age, had a robust classifying accuracy (Fig 2 , A). The Youden index–based cutoff generated for the model was 0.07, with a 0.88 sensitivity, 0.89 specificity, 0.38 PPV, and 0.99 negative predictive value. Kaplan-Meier analysis showed a very significant difference in survival for patients below or above the model threshold (P < .0001 [Fig 2, B]). Internal validation of the final model was performed by bootstrap resampling, showing maintained high accuracy (optimism-adjusted AUROCC = 0.93) (see Table E2 in this article’s Online Repository at www.jacionline.org). The COR+12 calculator based on the model can be found online at https://utrero-rico.shinyapps.io/COR12_Score/, thus allowing for calculation of a patient’s probability of death (see the instructions at the end of this article).

Fig 3

Comparison of variables for survivors who did not require ICU admission, survivors who required ICU admission, and nonsurvivors. There were significant differences in most of the variables between the patient groups. The ICU survivors (blue) were similar to the nonsurvivors (red) for some variables such as CRP level and neutrophil count. For other variables, however, ICU survivors had values intermediate between those for survivors who did not require ICU admission (gray) and those for nonsurvivors.

Table IV

Sequential regression models for mortality prediction in COVID-19

Model	SpO2/FiO2 ratio	N/L ratio	LDH level	IL-6 level	Age	AIC	H-L	AUC	95% CI	Cutoff	Sensitivity	Specificity	PPV	NPV
1	Yes	No	No	No	No	190.63	.05	0.87	0.80-0.95	0.12	0.75	0.88	0.32	0.98
2	Yes	Yes	No	No	No	175.14	.41	0.91	0.84-0.97	0.07	0.86	0.83	0.27	0.99
3	Yes	Yes	Yes	No	No	169.43	.57	0.91	0.84-0.97	0.11	0.81	0.88	0.34	0.98
4	Yes	Yes	Yes	Yes	No	136.41	.92	0.93	0.87-0.99	0.13	0.81	0.92	0.44	0.98
5	Yes	Yes	Yes	Yes	Yes	129.28	.93	0.94	0.89-1.00	0.07	0.88	0.89	0.38	0.99

AIC, Akaike information criterion; AUC, area under the curve; HL, Hosmer-Lemeshow test P value; NPV, negative predictive value.

Model 1 included 497 patients and 36 events, model 2 included 495 and 36 events, model 3 included 486 patients and 36 events, model 4 included 443 patients and 33 events, and model 5 includes 443 patients and 33 events.

Fig 2

The mortality risk model accurately classified patients at risk of dying. A, The area under the curve (AUC) of the model was 0.94 (95% CI = 0.89-1.00); the optimal cutoff of 0.07 had a sensitivity of 0.88 and a specificity of 0.89. B, Kaplan-Meier analysis based on Youden index optimal cutoff showed a very different survival rate between the groups with low and high risk of death (P < .0001). Color shades represent the 95% CI. Time is indicated in days. C, The score obtained from the model in nonsurvivors (red) was significantly higher than that in survivors who required intensive care (blue) (P = .0001) and in survivors who did not require intensive care (gray) (P = .0001). Dashed line indicates the optimal cutoff for mortality (0.07).

Table E2

Internal validation of the model by bootstrapping

	Risk model	Optimism	Optimism-adjusted
AUROCC	0.944	0.017	0.927
Calibration intercept	0.000	0.121	–0.121
Calibration slope	1.000	0.096	0.904

The developed model detected, early after their arrival at the hospital, 88% of patients who subsequently died. Only 4 patients stratified by the model as having a low mortality risk died (12% of the deceased patients) (see Table E3 in this article’s Online Repository at www.jacionline.org). Three of these patients (75%) had relevant comorbidities for mortality due to COVID-19. At the time of data collection, more than half of the patient cohort had no respiratory distress (281 [56.1%]), 113 patients (22.6%) had mild ARDS, 36 (7.2%) had moderate ARDS, and 67 (13.4%) had severe ARDS (Table I). The initial respiratory function assessment was a good predictor of mortality (SpO2/FiO2 ratio AUROCC = 0.87); however, 12 patients with no ARDS or mild-to-moderate ARDS in this early assessment died. Most of these patients (73%) who died without initial severe ARDS would have been classified by the model as having a high mortality risk (see Fig E4 in this article’s Online Repository at www.jacionline.org).

Table E3

Mortality score in the 36 deceased patients and clinical characteristics of the patients who had a low mortality risk score but died

Initial respiratory function	High score	Low score	No score available	Characteristics of the deceased patients with a low score
No ARDS (n = 4)	2	1	1	Kidney transplantation, hypertension, dyslipidemia, previous thromboembolic event, smoking
Mild ARDS (n = 4)	3	1	—	No comorbidities
Moderate ARDS (n = 4)	3	1	—	Hypertension, dyslipidemia, obesity, benign prostatic hyperplasia, steatosis
Severe ARDS n = 24)	21	1	2	HIV-HCV coinfection

HCV, Hepatitis C virus.

Fig E4

Contribution of the model to prediction of patient mortality in addition to their initial respiratory function assessment. SpO2/FiO2 ratio and mortality score are represented for every deceased patient in the cohort, except for 3 of 36 patients who died (1 indicates no ARDS, and 2 indicates severe ARDS), and no score was available. Of the 11 patients without severe ARDS who died, 8 (73%) had a high mortality score in the beginning of their hospitalization. The lower limit of the color shade corresponds to the model cutoff 0.07.

Finally, we analyzed all of the studied variables regarding ICU admission. Patients were divided into 3 levels of severity: patients who were discharged from hospital without having needed for intensive care, patients who required ICU admission but survived, and patients who died either after ICU admission or not. There were significant differences between the severity groups with regard to all variables except for AST level (Fig 3 ). For some biomarkers, such as CRP level or neutrophil count, ICU patients had values similar to those of nonsurvivors, whereas for other biomarkers such as LDH level, lymphocyte count, or SpO2/FiO2 ratio, ICU patients had intermediate values between those of less severely ill discharged patients and nonsurvivors. Next, the capacity of the risk model to predict intensive care requirement was tested. The score was significantly higher in patients who required ICU admission than in survivors who did not require ICU admission (0.059 [interquartile range, 0.015-0.15] vs 0.007 [interquartile range, 0.002-0.024]; P < .0001) (Fig 4 , A). The AUROCC for ICU requirement was 0.82 (95% CI = 0.74-0.91), with an optimal cutoff for the model of 0.03, which had 0.77 sensitivity and specificity. The value of the model to further define disease severity and to assist in hospital resource planning was tested; a significant positive correlation was found between the model and length of hospital stay in survivors (R  = 0.12; P < .0001) (Fig 4, B).

Fig 4

The mortality risk model could also be applied to other severity estimates such as ICU requirement and length of hospital stay. A, The model was an acceptable predictor of ICU requirement, with an area under the curve (AUC) of 0.82 (95% CI = 0.74-0.91) and optimal cutoff of 0.03, with 0.77 sensitivity and 0.77 specificity. The risk score was significantly higher for patients who required ICU admission than the score for those who did not (P < .0001). Dashed line represents the optimal cutoff for ICU admission (0.03). B, There was a positive correlation between the model and length of hospital stay by survivors (P < .0001).

Discussion

We have described a predictive model to classify patients with a recent COVID-19 diagnosis according to their risk of death by using 5 early, easy-to-assess biomarkers. This study analyzed a large patient cohort at a hospital in Madrid, Spain, at the peak of the pandemic. Recently, a score to predict critical illness, and 2 scores to predict mortality, , in hospitalized patients with COVID-19 have been published, with all of them analyzing patients from China. These have studies included a large number of variables in their model, and in contrast to in the present work, some of the variables, such as radiographic abnormalities or dyspnea, are relatively subjective. Additionally, a new machine learning model showed that LDH level, lymphocyte frequency, and CRP level can predict patient mortality more than 10 days in advance.

The present cohort confirmed in a non-Asian population some of the previously identified risk factors (eg, age). The patients included in the study were mostly of white or Hispanic ethnicity. A higher mortality rate was found in white patients than in Hispanic patients, which could be partly explained by age difference (56 years [range 47-64 years] vs 48 years [range 41-54 years]; P < .0001). The main comorbidities associated with fatal outcome were hypertension, dyslipidemia, and obesity. Of note, the overall mortality rate of 7.2% during follow-up was lower than that reported by other hospital cohorts , and lower than the 23% mortality rate described in the first consecutive 521 patients with COVID-19 in our hospital (from the end of February to the beginning of March, 2020) (Lalueza et al personal communication, 2020). Because it includes patients with IL-6 level measurements, the present cohort is younger (52 vs 65 years) and with lower rate of comorbidities (eg, a hypertension prevalence of 27.7% vs 42%) than in the work reported in the aforementioned communication by Lalueza et al. Moreover, 195 patients in our cohort (38.9%) received tocilizumab; the potential effect of this and other treatments on the mortality rate is currently being studied. The median time from hospital admission to laboratory testing was 2 days. However, the laboratory test results had minimal or no correlation with the time from onset of illness (see Fig E1), suggesting that the variables included in the study correlated with the severity of the disease rather than with the time interval since onset of symptoms or hospital admission.

Patients who progress to severe forms of COVID-19 are thought to have hyperactivation of the immune system with uncontrolled release of inflammatory cytokines. Signaling of the proinflammatory cytokine IL-6 can be blocked by the humanized anti–IL-6 receptor mAb tocilizumab. We set up measurement of IL-6 level in patients with COVID-19 at our hospital as a biomarker for inflammation and to help guide the administration of tocilizumab together with other inflammation markers such as CRP level, lymphocyte count, and clinical evaluation results. In our cohort, IL-6 level was increased in nonsurvivors. This reinforces the evidence that the amount of circulating IL-6 is closely linked to the severity of COVID-19, , , which has been further confirmed in a recent meta-analysis and contradicts the results reported for smaller published cohorts in which there were no differences in IL-6 level between patients with mild and severe COVID-19 cases. , IL-6 level has also been associated with mortality risk in patients with ARDS who do not have COVID-19. Measuring IL-6 level and including it in the predictive model may not only be relevant for prognosis but could also improve therapeutic decision making. There is an increasing number of studies reporting a favorable outcome from blocking IL-6 signaling in patients with COVID-19, , 24, 25, 26 all of them without a randomized control group. The model described here could be used as a tool to classify patients into more biologically and clinically uniform subgroups before inclusion in randomized trials for tocilizumab or other drugs. Until results from clinical trials are available, the current model can be useful to indicate those patients who could potentially benefit most from immunomodulatory treatments. Besides measuring IL-6 level, measuring the level of other cytokines could offer deeper insight into the pathophysiology of the disease or prognostic value, and it could also be targeted for therapy intervention. Notably, blocking of IL-1 with anakinra is proving beneficial in reducing the need for invasive mechanical ventilation and mortality among patients with severe COVID-19.

CRP is commonly used in clinical practice for decision making. In our study, it showed a remarkably high sensitivity for mortality prediction (0.97); however, it dropped out of the model, possibly because of its strong correlation with the variables SpO2/FiO2 ratio, N/L ratio, LDH level, and IL-6 level (see Fig E5 in this article’s Online Repository at www.jacionline.org). In particular, CRP levels are influenced by IL-6 level, as its cellular transcription can be a direct result of IL-6 signaling. Regarding the 3 levels of disease severity in Fig 3, CRP level appeared to be an accurate biomarker for predicting disease severity defined as ICU requirement or death.

Fig E5

Correlation plot of all the variables assessed as potential predictive biomarkers of mortality due to COVID-19. Positive correlations appear in blue, and negative correlations appear in red. The size of the circle corresponds to the magnitude of the correlation. The cross indicates no correlation.

Zhou et al highlighted d-dimer, together with age and SOFA score, as an independent risk factor for mortality. Our study did not confirm the predictive value of d-dimer level (Table II). In the study by Zhou et al, 81% of nonsurvivors had a d-dimer level higher than 1000 ng/mL, whereas in our cohort only 13% of nonsurvivors had a level above that threshold. It is plausible that this biomarker classifies a subgroup of patients with coagulopathy complications during COVID-19 and this subgroup was less represented in our cohort. Of note, d-dimer level was measured in only 330 patients, and this might have prevented its significance as biomarker. Similarly, in our cohort the transaminase ALT level was significantly lower in nonsurvivors, in contrast with the levels in previous studies in which ALT was increased in ICU patients or nonsurvivors. , Lei et al described a delayed increase in ALT level in patients with COVID-19 compared with the increase in levels of other liver enzymes. This could partially explain why ALT level was not increased in nonsurvivors at the time of measurement.

The final model includes only 5 variables, namely, SpO2/FiO2 ratio, N/L ratio, LDH level, IL-6 level, and age, which are readily obtainable at hospitals and clinical laboratories and can be obtained in a short time. Their combination boosted the mortality predictive accuracy of the model to an AUROCC of 0.94. Nevertheless, 5 sequential regression models were developed, and all of them can be used on the basis of the available data. For example, in the absence of laboratory data, SpO2/FiO2 ratio alone could be used as a biomarker predictive of mortality (AUROCC = 0.87), with 0.75 sensitivity and 0.88 specificity. Similarly, in a setting without measurements of IL-6 level, model 3 could be used, increasing the predictive capacity (AUROCC = 0.91) with 0.81 sensitivity and 0.88 specificity. The already published scores of patients with COVID-19 do not include IL-6 level; however, we have provided data supporting the potential relevance of this cytokine in prognosis and personalized treatment.

The potential limitations of this study include a relatively low number of deceased patients for developing the mortality risk model from a single enter. In addition, patients with measurements of IL-6 level were selected for inclusion in the cohort, which might have biased the cohort toward a younger median age. Finally, laboratory tests were performed not at time of admission but rather with a median delay of 2 days after hospitalization, and this may have limited the predictive capacity of the model. Validation of these results in other cohorts and at hospital admission is needed.

The COVID-19 pandemic has posed an enormous challenge for health care professionals and the medical systems worldwide. We have developed a risk model that could allow early identification of patients at high risk of a fatal outcome. The early identification of prognostic factors and patient groups at high risk of COVID-19 mortality could contribute to guiding medical decisions, planning for hospital resources (including intensive care), and reducing the fatality rate of this new infection.

COR+12 calculator instructions

Go to https://utrero-rico.shinyapps.io/COR12_Score/. Introduce the following 5 parameters: SpO2/FiO2 ratio, N/L ratio, LDH level (U/L), IL-6 level (pg/mL), and age (in years). Click the Predict button. The probability of death will appear in the graph, represented as a square with its 95% CI. By positioning the mouse over the square, the specific values for each patient can be visualized. For any issues regarding use of the COR+12 calculator, please contact R.L.G. at the e-mail address rociolagunagoya@gmail.com.

This model provides a quantifiable risk of fatal outcome in patients with COVID-19, including patients without respiratory distress at the time of evaluation, which can help guide medical decisions.