CURB-65, qSOFA, and SIRS Criteria in Predicting In-Hospital Mortality of Critically Ill COVID-19 Patients; a Prognostic Accuracy Study.

Introduction: Outcome prediction of intensive care unit (ICU)-admitted patients is one of the important issues for physicians. This study aimed to compare the accuracy of Quick Sequential Organ Failure Assessment (qSOFA), Confusion, Urea, Respiratory Rate, Blood Pressure and Age Above or Below 65 Years (CURB-65), and Systemic Inflammatory Response Syndrome (SIRS) scores in predicting the in-hospital mortality of COVID-19 patients.
Methods: This prognostic accuracy study was performed on 225 ICU-admitted patients with a definitive diagnosis of COVID-19 from July to December 2021 in Tehran, Iran. The patients' clinical characteristics were evaluated at the time of ICU admission, and they were followed up until discharge from ICU. The screening performance characteristics of CURB-65, qSOFA, and SIRS in predicting their mortality was compared.
Results: 225 patients with the mean age of 63.27±14.89 years were studied (56.89% male). The in-hospital mortality rate of this series of patients was 39.10%. The area under the curve (AUC) of SIRS, CURB-65, and qSOFA were 0.62 (95% CI: 0.55 - 0.69), 0.66 (95% CI: 0.59 - 0.73), and 0.61(95% CI: 0.54 - 0.67), respectively (p = 0.508). In cut-off ≥1, the estimated sensitivity values of SIRS, CURB-65, and qSOFA were 85.23%, 96.59%, and 78.41%, respectively. The estimated specificity of scores were 34.31%, 6.57%, and 38.69%, respectively. In cut-off ≥2, the sensitivity values of SIRS, CURB-65, and qSOFA were evaluated as 39.77%, 87.50%, and 15.91%, respectively. Meanwhile, the specificity of scores were 72.99%, 34.31%, and 92.70%. Conclusions: It seems that the performance of SIRS, CURB-65, and qSOFA is similar in predicting the ICU mortality of COVID-19 patients. However, the sensitivity of CURB-65 is higher than qSOFA and SIRS.

1. Introduction:

The COVID-19 pandemic started as an array of acute respiratory illnesses in Wuhan, China (1, 2). Although most patients infected with COVID-19 have minor signs, those older or with underlying diseases are more likely to have poor outcomes that progress to acute respiratory distress syndrome or multiple organ failure (3).

Early and rapid detection to prevent death in critically high-risk COVID-19 patients depends on the effective primary diagnosis (4). Prognostic scores can aid in clinical decision-making. There were several scores that international guidelines had confirmed to use for acute infectious disease before the COVID-19 pandemic (5). The Quick Sequential Organ Failure Assessment (qSOFA) score, the Confusion, Urea, Respiratory Rate, Blood Pressure and Age Above or Below 65 Years (CURB-65) score, and the Systemic Inflammatory Response Syndrome (SIRS) criteria are examples of ordinary scoring systems assessing disease severity, utilized in management of pneumonia and sepsis patients' condition (6).

The qSOFA score originated from three parameters: systolic blood pressure ≤ 100 mmHg, respiratory rate ≥ 22 breaths/min, and altered mental status (GCS<15) (4). The qSOFA simplifies the risk classification in patients with COVID-19 and has been broadly sued to screen patients with life-threatening conditions in hospitals (7). The CURBE-65 score involves vital signs such as Confusion, Urea> seven mmol/L, Respiratory Rate≥30 breaths/min, Blood Pressure (systolic<90mmHg, diastolic≤60mmHg), and age ≥65, and is used to predict the 30-day mortality in adult patients with bacterial pneumonia (8, 9). SIRS identifies with at least two from 4 following criteria: fever >38.0°C or hypothermia <36.0°C, tachycardia >90 beats/minute, tachypnea >20 breaths/minute, white blood cell count (<4×109/l or >12×109/l) (10).

Current evidence from developed countries approve of the utility of the qSOFA, CURB-65, and SIRS for predicting the outcomes related to COVID-19 patients, including the need for ICU admission or mortality (8, 11-13). However, such data is limited in developing countries (14).

Also, very few studies are available to identify whether these scores can be used to predict COVID-19 patients' mortality in ICU (14). Accordingly, this study aimed to compare the screening performance of qSOFA, CURB-65, and SIRS criteria in predicting the in-hospital mortality of ICU-admitted COVID-19 patients.

2. Methods:

This prognostic accuracy study was conducted in the Shahid Modarress Educational Hospital in Tehran, Iran, from July to December 2021. The scores of qSOFA, CURB-65, and SIRS criteria were calculated for all patients and the screening performance of models in predicting in-hospital mortality were compared. The participants were informed about the objectives, and patients or their relatives signed the informed consent forms. The study protocol was approved by the Ethics Committee of the clinical development unit of Loghman Hakim Hospital, Shahid Beheshti University of Medical Sciences, Tehran, Iran (IR.SBMU.RETECH.REC.1400.193; Research proposal cod:27784).

The study population included 225 patients with a definitive diagnosis of COVID-19 infection (based on Reverse transcription polymerase chain reaction (RT-PCR) technique) who were admitted to the ICU. Patients were selected using the convenience sampling technique. The patients were chosen according to the chest CT scan, PCR test, and doctor's approval. Inclusion criteria included patients admitted to ICU due to COVID-19, over 18 years old, and not pregnant. The exclusion criteria were transferring patients to other centers.

Research tools included a checklist containing demographic information (age, gender), baseline variables such as underlying diseases, blood test results, vital signs, lengths of hospitalization in the ICU, and discharge time. A checklist containing the parameters of qSOFA, SIRS, and CURB-65 was used to evaluate patients on admission to the ICU. The discharge status (mortality/survival) and patients' length of stay were also specified in this checklist. Accurate devices were used to measure the patient's vital signs, such as blood pressure (Hand pressure gauge), oxygen saturation level and heart rate (Calibrated pulse oximeter device), and temperature (non-contact thermometer). All patients were examined on admission and followed up until discharge from the ICU. The mortality status of patients in ICU was considered as the outcome. The patients' survival status at the time of discharge was recorded.

Analyses were performed using STATA 14.0 statistical software. Descriptive statistics included means ± SDs for quantitative variables and frequency (percentage) for qualitative variables. The independent sample t-test and Fisher's exact test were carried out to compare the variables between survivors and non-survivors. The area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, specificity, positive and negative likelihood ratios, and positive and negative predictive values with 95% confidence interval were calculated and reported for scoring systems (qSOFA, CURBE-65, and SIRS). Two cut-off values for each scoring system were used. We used a pre-defined cut-off of 2 for the evaluated tools based on previous studies (5, 13, 16, 17) and a cut-off of 1 according to the receiver operating characteristic (ROC) analysis of the current research.

3. Results:

225 patients with the mean age of 63.27±14.89 years were studied (56.89% male). Table 1 shows the baseline characteristics of studied cases. The in-hospital mortality rate of this series of patients was 39.10%. The mean age of survivors was significantly lower than the non-survived group (58.96±14.59 vs. 69.96±12.78 years; p < 0.001). The prevalence of hypertension (63.64 vs. 49.64%; p = 0.039) and diabetes (42.05 vs. 24.09%; p < 0.001) were significantly higher in non-survived group. The means respiratory rate (24.29 ± 4.88 vs. 22.43 ± 4.23 /minutes; p = 0.003) and blood urea nitrogen (64.40± 32.71 vs. 48.97± 28.10 mg/dl; p < 0.001) in the non-survived group were higher than in survivors, whereas the mean oxygen saturation was lower in the non-survived group (89.32 ± 6.14 vs. 85.49 ± 6.28 %; p < 0.001).

Table 1

Baseline characteristics of ICU-admitted COVID-19 patients

Variables	All patients (n= 225)	Survived(n=137)	Died (n=88)	P
Age (year)	63.27±14.89	58.96±14.59	69.96±12.78	<0.001
Gender
Male	128 (56.89)	78 (56.93)	50 (56.82)	0.986
Female	97 (43.11)	59 (43.07)	38 (43.18)
Hypertension
Yes	124 (55.11)	68 (49.64)	56 (63.64)	0.039
No	101 (44.89)	69 (50.36)	32 (36.36)
Diabetes mellitus
Yes	70 (31.11)	33 (24.09)	37 (42.05)	<0.001
No	155 (68.89)	104 (75.91)	51 (57.95)
Cardiovascular disease
Yes	62 (27.56)	34 (24.82)	28 (31.82)	0.251
No	163 (72.44)	103 (75.18)	60 (68.18)
Pulmonary disease
Yes	13 (5.78)	7 (5.11)	6 (6.82)	0.592
No	212 (94.22)	130 (94.89)	82 (93.18)
Kidney disease
Yes	17 (7.56)	7 (5.11)	10 (11.36)	0.083
No	208 (92.44)	130 (94.89)	78 (88.64)
Addiction
Yes	4 (1.78)	3 (2.19)	1 (1.14)	0.56
No	221 (98.22)	134 (97.81)	87 (98.86)
Malignancy
Yes	14 (6.22)	8 (5.84)	6 (6.82)	0.767
No	211 (93.78)	129 (94.16)	82 (93.18)
Others comorbidities
Yes	60 (26.67)	33 (24.09)	27 (30.68)	0.275
No	165 (73.33)	104 (75.91)	61 (69.32)
Vital signs
Pulse rate (/minutes)	82.74±16.91	81.54± 15.55	84.61±18.76	0.092
Respiratory rate (/minutes)	23.16±4.57	22.43±4.23	24.29±4.88	0.003
Temperature (°C)	37.08±0.31	37.05±0.29	37.13±0.33	0.071
SBP (mmHg)	129.78±20.98	128.65±19.04	131.54±23.70	0.314
DBP (mmHg)	78.72±13.78	77.89±13.31	80.02±14.47	0.258
Oxygen saturation (%)	87.82±6.45	89.32±6.14	85.49±6.28	<0.001
Laboratory results
Blood urea nitrogen (mg/dl)	55± 30.83	48.97±28.10	64.40± 32.71	<0.001
WBC (109 /l)	9.15±4.96	9.05±4.61	9.30±5.47	0.709
Length of stay in ICU (days)
Mean ± SD	12.23±7.81	12±7.45	12.60±8.37	0.593

Data are presents as mean ± standard deviation (SD) or frequency (%). All measures (except for length of stay) were evaluated at the time of admission to intensive care unit (ICU). P-values are obtained from independent sample t- test and chi-square test. SBP: Systolic blood pressure; DBP: Diastolic blood pressure; WBC: White blood cell count.

Table 2 displays the screening performance characteristics of SIRS, CURB-65, and qSOFA in predicting the in-hospital mortality of critically ill COVID-19 cases. The AUC of SIRS, CURB-65, and qSOFA were 0.62 (95% CI: 0.55 - 0.69), 0.66 (95% CI: 0.59 - 0.73), and 0.61(95% CI: 0.54 - 0.67), respectively (p = 0.508; Figure1). In cut-off ≥1, the estimated sensitivity values of SIRS, CURB-65, and qSOFA were 85.23%, 96.59%, and 78.41%, respectively. The estimated specificity of scores were 34.31%, 6.57%, and 38.69%, respectively. In cut-off ≥2, the sensitivity values of SIRS, CURB-65, and qSOFA were 39.77%, 87.50%, and 15.91%, respectively. Meanwhile, the specificity of scores were 72.99%, 34.31%, and 92.70%.

Table 2

Screening performance characteristics of studied scoring systems in predicting the in-hospital mortality of critically ill COVID-9 cases

Score	AUC	Sensitivity	Specificity	PPV	NPV	PLR	NLR
SIRS
≥ 1	0.62(0.55, 0.69)	85.23(75.70, 91.60)	34.31(26.55, 42.96)	45.45(37.75, 53.37)	78.33(65.47, 87.53)	1.29(1.11, 1.50)	0.43(0.25, 0.73)
≥2		39.77(29.66, 50.78)	72.99(64.62, 80.05)	48.61(36.78, 60.59)	65.36(57.20, 72.74)	1.47(1.01, 2.14)	0.82(0.69, 0.98)
CURB-65
≥ 1	0.66(0.59, 0.73)	96.59(89.66, 99.11)	6.57(3.24, 12.46)	39.91(33.34, 46.84)	75(42.83, 93.31)	1.03(0.97, 1.09)	0.51(0.13, 1.93)
≥ 2		87.50(78.32, 93.30)	34.31(26.55, 42.96)	46.11(38.43, 53.96)	81.03(68.18, 89.71)	1.33(1.15, 1.53)	0.36(0.20, 0.65)
qSOFA
≥ 1	0.61(0.54, 0.67)	78.41(68.10, 86.18)	38.69(30.60, 47.41)	45.10(37.12, 53.32)	73.61(61.68, 82.98)	1.27(1.07, 1.51)	0.55(0.36, 0.85)
≥ 2		15.91(9.28, 25.60)	92.70(86.64, 96.25)	58.33(36.94, 77.20)	63.18(56.10, 69.78)	2.17(1.01, 4.68)	0.90(0.82, 0.99)

All measures are presented with 95% Confidence interval; PPV: Positive predictive value; NPV: Negative predictive value; PLR: Positive likelihood ratio; NLR: Negative likelihood ratio; SIRS: Systemic Inflammatory Response Syndrome; CURB-65: Confusion, Urea, Respiratory Rate, Blood Pressure and Age Above or Below 65 Years; qSOFA: Quick Sequential Organ Failure Assessment.

Figure 1

Comparison of the area under the receiver operating characteristic (ROC) curve of SIRS, CURB-65, and q SOFA in predicting the inhospital mortality of intensive care unit-admitted COVID-19 patients (p = 0.508). SIRS: Systemic Inflammatory Response Syndrome; CURB-65: Confusion, Urea, Respiratory Rate, Blood Pressure and Age Above or Below 65 Years; qSOFA: Quick Sequential Organ Failure Assessment

4. Discussion:

We calculated the ROC curve results based on two cut-off values. The first cut-off (≥1) was obtained from our data, and the second one (≥2) was evaluated according to previous similar studies. The results of ROC curves indicated that the performance of the three assessed scoring systems had the same pattern. In cut-off≥1, the sensitivity value of CURB-65 (96.59%) was higher than SIRS (85.23%) and qSOFA (78.41%). Also, in cut-off≥2, the same results were observed; the sensitivity value of CURB-65 (87.50%) was higher than SIRS (39.77%) and qSOFA (15.91%).

CURB-65 is a clinical prediction rule intended to stratify pneumonia patients by expected mortality and has a major advantage in its simplicity (18). Also, the CURB-65 scores are an impressive example of the value of a simple clinical approach not requiring sophisticated biochemical, immunological, or genetic data in the risk stratification of patients with an acute, potentially life-threatening condition (19). Most of the studies related to COVID-19 showed that CURB-65 has good performance in predicting mortality in COVID-19 patients (18, 19).

According to our search, the only similar study was conducted in India to predict the mortality of 140 COVID-19 patients in ICU. The study illustrated that the NEWS2, SIRS, qSOFA, and CURB-65 were calculated prospectively, and the CURB-65 (AUC: 0.72) had better performance in predicting ICU mortality compared to qSOFA (AUC: 0.63) and SIRS (AUC: 0.54) (14).

In a multicenter prospective study with 830 adult patients with COVID-19, the prognostic scores, including CURB-65, NEWS2, and qSOFA, were compared. The AUC values of CURB-65 and qSOFA for ICU admission were 0.63 and 0.56, respectively. Also, in the low-risk levels (CURB-65 score <2 and qSOFA score<2), the mortality rates were 16.7% and 21.4%, respectively (5).

In line with the results of this study, there were several studies with COVID-19 patients that confirmed the same results. The AUC values of qSOFA were 0.56, 0.55, 0.69, and 0.78, indicating its inferior performance compared with other scoring systems (13, 20-22). Also, the AUC values of CURB-65 and SIRS were 0.76 and 0.74, indicating their lower performance than NEWS (13, 22).

On the other hand, in a recent retrospective cohort study with 247 COVID-19 patients admitted to ICU, the efficiency of CURB-65 in predicting 30-day mortality was assessed, and the findings indicated that among patients hospitalized with COVID-19 those with CURB-65≥ 2 had a higher risk of 30-day mortality (8).

Although one of the advantages of the qSOFA score is its simplicity and speed of application, several studies showed that the qSOFA score had a low sensitivity for in-hospital mortality of patients with suspected infection (4). Also, the results of a retrospective study with 140 severely ill COVID-19 patients illustrated that the area under the ROC curve for SOFA (0.89) was significantly higher than qSOFA (0.74). Thus, the performance of SOFA was superior to that of qSOFA (4). And also, an observational retrospective study with 237 adults hospitalized with COVID-19 was performed to compare the application of SOFA, qSOFA, and NEWS. Their results displayed that the AUC value of qSOFA was found to be 0.72 (cut-off ≥1) for in-hospital mortality outcomes and had an infirm performance compared to other scores. However, they noted that sepsis scores were of value for predicting respiratory failure and mortality in COVID-19 patients (11).

However, only one study showed that performance of q-SOFA in COVID-19 patients was better than SIRS and CURB-65. This retrospective cohort study with 235 elderly adults was conducted to compare the seven scoring systems in predicting the mortality rate. The results showed that the AUC of SIRS, CURB-65, and qSOFA were 0.69, 0.85, and 0.88, respectively (6).

5. Limitation

The small sample size and the lack of long-term follow-up for the survived patients after ICU discharge are among the limitations of the present study.

6. Conclusion:

The present study showed that the AUC values for CURB-65 were slightly higher than qSOFA and SIRS, but these differences were not significant, so the three evaluated scores had the same performance for predicting mortality in patients with COVID-19 admitted to ICU. However, the sensitivity of CURB-65 was higher than qSOFA and SIRS.

7. Declarations:

7.1. Acknowledgments

The authors express their gratitude to the Clinical Research Development Unit of Loghman Hakim Hospital, Shahid Beheshti University of Medical Sciences, for their support and cooperation during the study.

7.2. Funding sources

Shahid Beheshti University of Medical Sciences aided this study.

7.3. Competing interest

There are no conflicts of interest.

7.4. Authors' contributions

Study design: SK, MP, MY

Data gathering: AS, SK

Analysis: MY

Interpretation of results: MY, SK

Drafting of manuscript: SK, MP, MY

Critically revised: All authors.