Development of a clinical risk score to predict death in patients with COVID-19.

OBJECTIVE: To build a clinical risk score to aid risk stratification among hospitalised COVID-19 patients.
METHODS: The score was built using data of 417 consecutive COVID-19 in patients from Kuwait. Risk factors for COVID-19 mortality were identified by multivariate logistic regressions and assigned weighted points proportional to their beta coefficient values. A final score was obtained for each patient and tested against death to calculate an Receiver-operating characteristic curve. Youden's index was used to determine the cut-off value for death prediction risk. The score was internally validated using another COVID-19 Kuwaiti-patient cohort of 923 patients. External validation was carried out using 178 patients from the Italian CoViDiab cohort.
RESULTS: Deceased COVID-19 patients more likely showed glucose levels of 7.0-11.1 mmol/L (34.4%, p < 0.0001) or >11.1 mmol/L (44.3%, p < 0.0001), and comorbidities such as diabetes and hypertension compared to those who survived (39.3% vs. 20.4% [p = 0.0027] and 45.9% vs. 26.6% [p = 0.0036], respectively). The risk factors for in-hospital mortality in the final model were gender, nationality, asthma, and glucose categories (<5.0, 5.5-6.9, 7.0-11.1, or 11.1 > mmol/L). A score of ≥5.5 points predicted death with 75% sensitivity and 86.3% specificity (area under the curve (AUC) 0.901). Internal validation resulted in an AUC of 0.826, and external validation showed an AUC of 0.687.
CONCLUSION: This clinical risk score was built with easy-to-collect data and had good probability of predicting in-hospital death among COVID-19 patients.

INTRODUCTION

Severe acute respiratory syndrome coronavirus 2 (SARs‐CoV‐2), causing coronavirus disease 2019 (COVID‐19), is currently the greatest public health threat in the world. Since its emergence at the end of 2019, it has spread globally and resulted in the World Health Organization (WHO) categorising it as a worldwide pandemic in 2 March020. , Although this is not the first coronavirus to infect the human population (SARs‐CoV‐1 and MERs), the velocity of SARs‐CoV‐2 transmission differentiates it from other viruses. Furthermore, its ability to result in fatal disease and acute respiratory distress syndrome (ARDS) necessitates the development for effective treatment and prevention strategies. , ,

Individuals with COVID‐19 vary from asymptomatic to critically severe cases that lead to ARDS, intensive care unit (ICU) admissions, invasive mechanical ventilation, and mortality. , Across borders, severe cases of COVID‐19 have been seen in patients who are predominantly male, older than 65 years, and have one or more comorbidities, with hypertension, diabetes, and cardiovascular disease (CVD) being the most pertinent. , , , , Increased risk in individuals with comorbidities is possibly due to the mode in which SARs‐CoV‐2 infects cells and spreads within the body. SARs‐CoV‐2 binds via its Spike (S) protein to the angiotensin‐converting enzyme 2 (ACE2) receptor, facilitating its entry into cells of the respiratory tract. It has been reported that SARs‐CoV‐2 has a 10‐ to 20‐fold higher binding affinity to ACE2 than previous coronaviruses (namely SARs‐CoV‐1), hence increasing the uptake of SARs‐CoV‐2 and aiding in its increased pathogenicity. , ,

While many studies have reported the clinical findings of hospitalised COVID‐19 patients, hospitals and healthcare staff globally remain overwhelmed. Though the development of vaccines against COVID‐19 , , , signals hope for controlling the pandemic, tens of thousands of new cases are being reported every day and hospital staff need to be able to predict which patients are more likely to succumb to a severe form of COVID‐19, ARDS, ICU admission, or even death.

In this current study, we attempt to build a clinical risk score to aid clinicians in identifying patients more likely to develop critical cases of COVID‐19 to better optimise care. Additionally, the development of new therapies with limited access to the general population, such as tocilizumab, might benefit from such a risk score. The score was built using clinical data of a COVID‐19 cohort from Kuwait and then validated with an external cohort of Italian COVID‐19 patients (CoViDiab).

METHODS

Data collection

To build the clinical risk score, we used data collected retrospectively from 417 consecutive patients positive for COVID‐19 who were hospitalised at Jaber Al‐Ahmad hospital (Kuwait) between February 24th and 3 May 2020. At a time when the local policy was to admit anyone with a positive COVID‐19 real‐time polymerase chain reaction (RT‐PCR) of a nasopharyngeal swab regardless of symptom status. Hence, the population included in the analysis ranged from asymptomatic to severe cases.

Asymptomatic patients were defined as patients who had an RT‐PCR positive for COVID‐19, but who presented with no symptoms and did not require ICU treatment. On the other hand, symptomatic patients were those who had mild to moderate symptoms typical of COVID‐19. These were patients who could still be treated in the wards and did not require ICU admission. Severe cases were characterised as those that require ICU admission, mechanical ventilation, and includes those that lead to mortality.

Statistical analysis

Descriptive statistics are presented for categorical variables as numbers with proportions and for continuous variables as appropriate measures of central tendency and dispersion. Student's t‐test was used to compare differences in continuous variables between groups, categorical variables were compared with a X2 test. The primary outcome was defined as in‐hospital death. Multivariate logistic regression models were then used to identify independent prognostic factors for the primary outcome. They were built step‐by‐step by adding or removing variables based on the results of previous models and retaining in the final model variables associated with the outcome of a nominal p‐value <0.1. Before entering in the model, continuous variables (age and blood glucose) were converted into ordinal variables based on recognized cut‐offs (age: <50, 51–70, and 70 > years of age; blood glucose: <5.5, 5.5–6.9, 7.0–11.1, and 11.1 > mmol/L).

Briefly, we initially tested comorbidities such as hypertension, diabetes, malignancy, chronic renal disease, and asthma against the primary outcome via logistic regressions. Then we performed a similar logistic regression using demographic information. The predictive variables that proved to be significant at the nominal p‐value <0.1 were carried out by performing additional regressions merging the data. The final model included gender, non‐Kuwaiti national, asthma, and glucose categories as the predictive variables. Significant risk factors were assigned weighted points that were proportional to their beta regression coefficient values. The reference group of categorical variables were assigned 0 points, corresponding to a beta‐coefficient of zero. Receiver‐operating characteristic (ROC) curve analyses were performed to assess the effectiveness of our risk score to predict death in patients hospitalised for COVID‐19. In terms of disease prediction, typically an area under the curve (AUC) of 0.5 or less is considered insignificant, an AUC of 0.7–0.8 is considered an acceptable fit, meaning that the score can somewhat be able to predict patients more likely to proceed to the main outcome, and an AUC of 0.9 or above is a great fit for the score, indicating that the score is able to predict the outcome with a high degree of confidence. The cut‐off for death prediction was determined using Youden's index. Two‐tailed p‐value <0.05 was considered as statistically significant in all analyses. All statistical analyses were performed with SPSS (IBM Corp. Released 2012. IBM SPSS Statistics for Windows, Version 21.0. Armonk, NY: IBM Corp.).

Internal and external validation cohort

The score was both internally and externally validated. Internal validation was performed by calculating the total clinical risk score for each patient within a separate COVID‐19 cohort from Kuwait (admitted between May 4th and 26 August 2020). Patient data was obtained retrospectively, and inclusion criteria in the validation cohort was based on the presence of admission data and availability of discharge information (either dead or alive). Patients lacking this information were excluded from the validation cohort. The respective scores were tested against the main outcome and analysed by an ROC, the “goodness of fit” of the score was determined by the AUC of the ROC curve.

The cohort used for external validation of the score was composed of patients from the Italian CoViDiab cohort. As previously described, CoViDiab is a multi‐center observational study collecting data retrospectively from medical charts of patients hospitalized for COVID‐19 in four academic hospitals located in the Lazio region of Italy up to 15 May 2020. Patients eligible for inclusion were aged ≥18 years old with a diagnosis of COVID‐19 confirmed by at least one RT‐PCR in agreement with the protocol set by the WHO. All the clinical data needed to calculate the proposed score and discharge information (either dead or alive) were available in 178 of the 354 patients originally enrolled in the CoViDiab study.

Ethical considerations

Ethical approval was obtained from the standing committee for coordination of health and medical research at the Ministry of Health in Kuwait (IRB 2020/1404). The necessity of written informed consent was waived by the standing committee for coordination of healthcare and medical research at the Ministry of Health in Kuwait due to the urgent need for data collection and the nature of the disease under investigation.

CoViDiab complies with the principles of the Helsinki Declaration and was approved by the Ethical Committee of Umberto I “Policlinico” General Hospital (ref. 5819/2020). Due to the study's retrospective design, informed consent was waived in cases of discharge, of impossibility of contact with patients, and in case of death. The privacy and anonymity of the data collected were guaranteed in agreement with current regulations.

Data availability

The data of the 417 patients underlying the results presented in the study are available from https://doi.org/10.6084/m9.figshare.12567881.v1. ,

RESULTS

Demographic characteristics

The demographic characteristics of the 417 patients of the Kuwaiti COVID‐19 cohort are shown in Table 1. The cohort was divided into those who developed the main outcome (death) versus those who did not. Initially, those who developed the outcome were older than those who did not (54.2 vs. 43.9 years). When looking at glucose values, those who proceeded to the outcome were more likely to have glucose levels >11.1 mmol/L than those who did not (44.3% vs. 5.0%, p < 0.0001). Comorbidities, such as diabetes, hypertension, and asthma were more prevalent in patients dying from COVID‐19 than those who survived (39.3% vs. 20.4% [p = 0.0016], 45.9% vs. 26.6% [p = 0.0033], 19.7% vs. 8.1% [p = 0.0085]).

TABLE 1

Demographic characteristics of 417 patients of the Kuwaiti COVID‐19 cohort

Primary outcome (death)
Variable	No (n = 357)	Yes (n = 60)	p‐value	95% confidence interval
Age, mean years ± SD	43.8 ± 17.50	53.6 ± 12.2	<0.0001
Male gender, n (%)	208 (58.3)	54 (90.0)	<0.0001	0.06459, 0.3675
Kuwaiti, n (%)	228 (63.9)	12 (20.0)	<0.0001	3.596, 13.69
Blood glucose categories (mmol/L)
<5.5, n (%)	179 (50.1)	5 (8.3)	<0.0001	4.326, 28.29
5.5‐6.9, n (%)	113 (31.7)	7 (11.7)	0.0011	1.545, 7.956
7.0‐11.1, n (%)	47 (13.2)	21 (35.0)	<0.0001	0.1525, 0.5197
>11.1, n (%)	18 (5.0)	27 (45.0)	<0.0001	0.03237, 0.1301
Comorbidities
Diabetes, n (%)	73 (20.4)	24 (40.0)	0.0016	0.2165, 0.6867
Hypertension, n (%)	95 (26.6)	28 (46.7)	0.0033	0.2369, 0.7248
CVD, n (%)	26 (7.3)	13 (21.7)	0.0013	0.1365, 0.5909
Asthma, n (%)	29 (8.1)	12 (20.0)	0.0085	0.1691, 0.7397
Malignancy, n (%)	9 (2.5)	3 (5.0)	0.3952	0.1291, 1.870
ICU admission, n (%)	22 (6.2)	60 (100.0)	<0.0001	3.315e‐005, 0.009266

Note: The table shows the characteristics of COVID‐19 patients who developed the main outcome versus those who did not. p‐values were calculated using Fisher's exact t‐test.

Abbreviation: CVD, cardiovascular disease.

The demographic characteristics of the original Kuwaiti cohort, compared with the demographic characteristics of the validation cohorts, are shown in Table 2.

TABLE 2

Demographic characteristics for COVID‐19 patients across all cohorts

	Kuwait COVID‐19 cohort (417)	Kuwaiti internal validation cohort (923)	CoViDiab population (Italy) (178)
Age (mean ± SD), years	45.38 ± 17.07	48.34 ± 19.43	63 [54–77] a
Gender	262 (62.8%)	536 (58%)	106 (59.6%)
Non‐Kuwaiti	177 (42.4%)	219 (23.7%)	178 (100%)
BMI (kg/m2) b
≤25	‐	67 (7.3%)	41 (23.0%)
25–29.9	‐	96 (10.4%)	42 (23.6%)
≥30	‐	157 (17.0%)	95 (53.4%)
Glucose (mmol/L)
<5.5	184 (44.1%)	272 (29.4%)	64 (36.0%)
5.5–6.9	120 (28.8%)	268 (29.0%)	54 (30.3%)
7.0–11.1	68 (16.3%)	229 (24.7%)	49 (27.5%)
>11.1	45 (10.8%)	141 (15.3%)	11 (6.2%)
Comorbidities
Hypertension	123 (29.5%)	199 (21.5%)	92 (51.7%)
Diabetes	97 (23.3%)	81 (8.8%)	39 (21.9%)
Dyslipidaemia	‐	‐	39 (21.9%)
CVD	39 (9.4%)	‐	23 (12.9%)
Asthma	41 (9.8%)	42 (4.5%)	‐
Malignancy	12 (2.9%)	24 (2.6%)	8 (4.5%)
ICU admission	82 (19.7%)	166 (18.0%)	24 (17.4%)
Death	60 (14.4%)	121 (13.1%)	21 (11.8%)

Abbreviations: BMI, body mass index; CVD, cardiovascular disease.

Median age [Interquartile range], the CoViDiab cohort was not normally distributed and thus median age was determined.

BMI information was not available for all patients.

In the 417 patients of the COVID‐19 Kuwaiti cohort, the patients had a mean age of 45.38 ± 17.07, 19.7% of the cohort was admitted to the ICU, with 14.4% resulting in the primary outcome (death). The internal validation Kuwaiti cohort consisted of 923 patients with a mean age of 48.34 ± 19.43 years, with 18.0% of the cohort admitted to the ICU, and 13.1% resulting in the primary outcome, showing a similar trend in the initial Kuwaiti cohort. The median age of the CoViDiab cohort from Italy was 63 years, with 17.4% of the patients being admitted to the ICU and 11.8% resulting in the primary outcome.

Developing the score

Nationality, gender, asthma, and blood glucose levels were the predictive variables independently associated with the primary outcome. Each variable was allocated a specific score based on the calculated beta coefficients of each predictive variable, as shown in Table 3.

TABLE 3

Calculated clinical risk score. Low risk of progression is a total clinical risk score of <5.5, a higher risk of progressing to the main outcome (death) is a score of ≥ 5.5

Criteria	Score
Male	2.5
Non‐Kuwaiti national	2.5
Asthma	2.5
Blood glucose 7.0–11.1 mmol/L	3.5
Blood glucose ≥ 11.1 mmol/L	5.0

Note: This was calculated based on the Youden's index of the score (the point on the ROC curve that retains high sensitivity and 1‐specificity).

Abbreviation: ROC, receiver‐operating characteristic.

The maximum allocated score was 12.5. The cut‐off value to predict death was 5.5, which showed a sensitivity of 75% and specificity of 86.3% to predict the outcome (AUC 0.901).

Internal and external validation

The score was internally and externally validated in order to assess its predictive potential (Table 4, Figure S1). Internal validation was performed on two cohorts from the COVID‐19 population in Kuwait. The first population included the original 417 patients used to build the score, and the second cohort included 923 COVID‐19 patients admitted between May 4th and 26 August 2020, in one designated COVID‐19 centre in Kuwait.

TABLE 4

Comparison of score results within different cohorts. Kuwaiti cohorts were used for internal validation of the score, these are Kuwaiti COVID‐19 (417 patients) cohort and Kuwaiti COVID‐19 (923)

Cohort	% Sensitivity	%Specificity	AUC ± SE	PPV	NPV
Kuwaiti (417)	75.0	86.3	0.901 ± 0.20	47.8%	95.4%
Kuwaiti (923)	66.9	76.7	0.826 ± 0.91	30.2%	93.9%
CoViDiab (178)	66.7	70.7	0.687 ± 0.06	23.3%	94.1%

Note: The Italian CoviDIAB (178 patients) cohort was used for external validation. %Sensitivity and %Specificity were derived from the ROC analysis using the Youden's index (5.5). AUC represents the AUC, an AUC of 0.9–1.0 is an excellent fit for the model, 0.8–0.7 is a great fit, and 0.6 indicates a good fit. Using a cut‐off score of 5.5, PPV and NPV were calculated based on the following formula: PPV = True positive/True Positive + False Positive, NPV = True Negative/True Negative + False Negative.

Abbreviations: AUC, area under the curve; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver‐operating characteristic.

Internal validation of the cohort showed that within the 417 patients, the AUC was 0.901, indicating that the clinical risk score had good predictive value. Further internal validation utilising a separate COVID‐19 cohort from Kuwait resulted in an AUC of 0.826, signifying an acceptable fit for the score. The score was also externally validated using a patient cohort from Italy (CoViDiab, 178 patients). The score was calculated for each patient and then tested against the main outcome (death); the results of the external validation showed an AUC of 0.687, indicating a good fit for the score.

DISCUSSION

This study developed a clinical risk score for the prediction of severe disease and death in COVID‐19 patients. The score was developed retrospectively, utilising data from 417 consecutive patients hospitalised in one COVID‐19 centre in Kuwait. The score was based on assessing clinical and comorbid data from these patients, focussing on clinical data that would be routinely collected in any hospital or health centre internationally. The Kuwaiti COVID‐19 cohort used to build the clinical risk score was symptomatically diverse. This was primarily due to the initial steps the Kuwaiti government had taken within the first outbreak of COVID‐19 in the country. These patients were recruited at a time when there was a 100% hospitalisation protocol in place for anyone with one positive PCR for SARs‐CoV‐2.

Hence, the cohort used to build the score consisted of patients ranging from asymptomatic to those with severe symptoms, giving a broad idea of the pathophysiology of the virus and who is more at risk. Compared with other reports that have attempted to build predictors for COVID‐19 severity and death, the variation of patient characteristics in our cohort gives added value to understand how the disease progresses. , Moreover, few risk scores have been developed utilising data from Middle Eastern (ME) populations. This leaves a gap for the specific ethnic variation between the ME and European populations, which could influence disease susceptibility. Hence, externally validating our score with the CoViDiab cohort, a completely different patient demographic, supports the predictive capabilities of our developed score and its efficacy among different populations.

The Kuwaiti cohort had a mean age of 45.38 ± 17.07, with around 62.8% of the cohort being male patients. Roughly 23.4% of the cohort was diabetic and 29.5% was hypertensive (Table 2); this reflects the current prevalence of these conditions within the general Kuwaiti population. Studies have shown that Kuwait has a 23% and 42% prevalence of diabetes and hypertension, respectively, within the general population. These conditions, along with obesity, CVD, and other comorbidities have been continuously cited as linked to poorer outcome and increased mortality in COVID‐19 patients. Reports have suggested that these individuals are more at risk due to the pathophysiology of their underlying conditions. For instance, individuals with diabetes, especially diabetic patients with uncontrolled hyperglycaemia, have compromised innate and humoral immune systems. Diabetes can attribute to a proinflammatory state; thus, when diabetic patients contract COVID‐19, it has been reported that they have a significant increase in systemic levels of C‐ reactive protein and interleukin‐6 (IL‐6). In addition, increased recruitment of T helper cells, triggering an already exacerbated inflammatory response and increased production of interferon gamma, results in a cytokine storm. ,

The final model for the score included being male, non‐Kuwaiti national, having asthma, blood glucose between 7.0 and 11.1 mmol/L and glucose levels greater than 11.1 mmol/L. When tested within the primary cohort, the ROC curve had an AUC of 0.901 with an Negative predictive value (NPV) of 95.4%, indicating a great fit for the curve with a high probability to distinguish those without the primary outcome from those with the outcome. In our cohort, being non‐Kuwaiti may attribute to the primary outcome due to the socioeconomic differences present. Non‐Kuwaitis were predominantly of South Asian descent, these individuals are more likely to be male, laborers, and living in tight quarters. Hence, increasing their susceptibility to contracting SARs‐CoV‐2.

We saw that the addition of hyperglycaemia was a pushing factor for severe outcome regardless of the diabetic state. In fact, hyperglycaemia retained its significance, while diabetes and hypertension lost theirs. Alshukry et al., have gone on to suggest that when it comes to COVID‐19 severity, fasting blood glucose (FBG) may play a key role in assessing disease outcome. The authors have stated that there is a non‐linear relationship to ICU admission and increased hyperglycaemia, even saying that an increase in FBG from 6.9 mmol/L to 7.0 mmol/L was met with a 15‐fold higher odds ratio of ICU admission. , This further imposes the importance of glucose monitoring within COVID‐19 patients. It is important to note that glucose levels included in this study, across both the Kuwaiti and the CoViDiab cohorts, were random glucose measures. This was due to the data being collected at the height of the pandemic, and it being infeasible to collect fasting glucose measurements, especially for ICU patients. Moreover, this aided in standardising the results for the study. Additionally, corticosteroids were used in 44.3% of the 178 CoViDiab participants included in this analysis. Patients treated with corticosteroids were 6.5 [95% CI: 2.09–20.26] times more likely to die in comparison with inpatients not treated with corticosteroids. However, biologicals and steroids were not initially used in the Kuwaiti cohorts, this was due to there being conflicting results on the use of steroids in the beginning of the pandemic.

Furthermore, hyperglycaemia and chronic inflammation are already well established, especially in terms of diabetic complications. Studies have shown that patients admitted with COVID‐19 and abnormal FBG levels were typically older and presented with more underlying conditions than those with normal FBG (56.44 ± 11.64 years, 57.6% vs. 39.55 ± 16.59 years, 14.7%, p < 0.001, respectively). Additionally, glycosylation of the ACE2 receptor has been demonstrated to increase binding affinity of the virus. Uncontrolled hyperglycaemia can cause continued glycosylation of the ACE2 receptor, which not only can lead to continued infiltration of SARs‐CoV‐2 but may allow for increased severity by promoting widespread organ susceptibility.

In the internal and external validation cohorts, the AUC for the Kuwaiti internal validation cohort was 0.826 and for the CoViDiab cohort was 0.687, with an NPV of 93.0% and 94.1%, respectively (Table 3). The high NPV values are important in the prediction of disease outcome; this value is able to distinguish between true and false negatives. Thus, suggesting that if a patient is identified as having a lower risk of succumbing to the main outcome with our developed score, we can have a good degree of confidence that this is a true negative.

Study limitations

The retrospective nature of this study made it difficult to obtain data that was lacking, such as BMI information and HbA1C. This missing data may add critical information that may impact the development of severe COVID‐19 and our clinical risk score. Furthermore, it is important to note that glucose management and treatment differences among our internal and external validation cohorts may also impact results. It is also important to note that information for asthma was missing from the CoViDiab population. Another limitation is the fact that the CoViDiab population is mainly focussed on diabetic patients unlike the general population admitted to hospitals. Nonetheless, given the high rate of diabetes, this population still constitutes a valid study population. Lastly, to deduce the true efficacy of the score, further external validation is required.

CONCLUSION

The proposed risk score built with easy‐to‐collect clinical data had good performance for predicting in‐hospital death among patients with COVID‐19.

CONFLICT OF INTEREST

The authors declare no competing interests.

ETHICS STATEMENT

Ethical approval was obtained from the standing committee for coordination of health and medical research at the Ministry of Health in Kuwait (IRB 2020/1404).

AUTHOR CONTRIBUTION

Ghadeer Alhamar was responsible for drafting the article, study design, data analysis and interpretation. Ernesto Maddaloni and Raffaella Buzzetti conceptualised the design of the study, were involved in data analysis and interpretation, and revising the manuscript. Abduallah A. Al‐Shammari, Abdullah Al Shukry, Mohamed Abu‐Farha, Jehad Abubaker, and Ebaa AlOzairi were involved in data collection of the Kuwaiti COVID‐19 cohort. Salman Al‐Sabah, Sulaiman Almazeedi, Mohannad Al‐Haddad, Sarah Al‐Youha, Mohammed Jamal, and Abdulnabi T Alattar were responsible in the data collection of the Kuwait internal validation cohort. Luca D’Onofrio contributed to the design of the study, data acquisition and revised the manuscript. Francesco Alessandri, Carmen Mignogna, Gaetano Leto, and Giuseppe Pascarella contributed to data acquisition and interpretation of data. Francesco Pugliese and Carlo Maria Mastroianni contributed to the design of the study, acquisition of the data and revision of the article. Hamad Ali was involved in data collection of the Kuwait COVID‐19 cohort, data analysis and interpretation and critical revision of the article. Fahd Al Mulla played a role in critical revision of the article. Paolo Pozzilli was involved in revision of the manuscript, as well as had final approval of the version to be published.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1002/dmrr.3526.

Supporting Information 1

Click here for additional data file.