Prognostic Accuracy of the SIRS, qSOFA, and NEWS for Early Detection of Clinical Deterioration in SARS-CoV-2 Infected Patients.

BACKGROUND: The case fatality rate of coronavirus disease 2019 (COVID-19) is estimated to be between 4.3% and 11.0%. Currently there is no effective antiviral treatment for COVID-19. Thus, early recognition of patients at high risk is important.
METHODS: We performed a retrospective observational study of 110 patients with severe acute respiratory syndrome coronavirus 2 infection. We compared the effectiveness of three scoring systems: the Systemic Inflammatory Response Syndrome (SIRS), quick Sequential Organ Failure Assessment (qSOFA), and National Early Warning Score (NEWS) systems, for predicting the prognosis of COVID-19. The area under the receiver operating characteristic curve (AUROC) was used for these assessments, and Kaplan-Meier survival curves were used to identify the cumulative risk for 28-day mortality according to the NEWS stratification.
RESULTS: For predicting 28-day mortality, NEWS was superior to qSOFA (AUROC, 0.867 vs. 0.779, P < 0.001), while there was no significant difference between NEWS and SIRS (AUROC, 0.867 vs. 0.639, P = 0.100). For predicting critical outcomes, NEWS was superior to both SIRS (AUROC, 0.918 vs. 0.744, P = 0.032) and qSOFA (AUROC, 0.918 vs. 0.760, P = 0.012). Survival time was significantly shorter for patients with NEWS ≥ 7 than for patients with NEWS < 7.
CONCLUSION: Calculation of the NEWS at the time of hospital admission can predict critical outcomes in patients with COVID-19. Early intervention for high-risk patients can thereby improve clinical outcomes in COVID-19 patients.

INTRODUCTION

In December 2019, an outbreak of coronavirus disease 2019 (COVID-19) from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection causing pneumonia began in Wuhan, China, and is rapidly spreading worldwide.1 By April 11, 2020, 1,610,909 cases of COVID-19 had been diagnosed, and 99,690 cases of COVID-19 patients had died. The Chinese Center for Disease Control and Prevention reported 81% mild, 14% severe, and 5% critical cases.2 The case fatality rate is estimated to be between 4.3% and 11.0%, lower than that of severe acute respiratory syndrome (SARS) and Middle East respiratory syndrome, but higher than that of influenza.3

There are several established clinical scoring systems for detecting patients at high risk for a suspected infection. The Systemic Inflammatory Response Syndrome (SIRS) score was developed during a 1991 consensus conference to improve early detection of sepsis in patients.4 SIRS consists of four variables: body temperature, heart rate, respiration rate, and white-blood-cell count with differentials. However, it has limited value for predicting mortality among infected adults in5 and outside6 an intensive care unit (ICU) setting.

The quick Sequential Organ Failure Assessment (qSOFA) criteria, consisting of the respiratory rate, Glasgow Coma Scale, and systolic blood pressure, were proposed in 2016. qSOFA is better at predicting in-hospital mortality than SIRS outside an ICU setting6 but it has lower prognostic accuracy for in-hospital mortality than does an increase in qSOFA score of 2 or more in an ICU setting.5

The National Early Warning Score (NEWS) consists of seven parameters to improve the early detection of and response to clinical deterioration.7 NEWS has equivalent or superior accuracy to that of SIRS and qSOFA for predicting clinical deterioration in infected patients outside the ICU.89

To date, there is no effective antiviral treatment for COVID-19. Thus, early recognition of patients at high risk is important, as is close monitoring to provide the necessary interventions at the right time. This study compared the predictive values of preexisting clinical scoring systems in patients with COVID-19.

METHODS

Study design and subjects

We performed a retrospective observational study of 110 patients with SARS-CoV-2 infection hospitalised at Yeungnam University Medical Center (a 930-bed, university-affiliated, tertiary referral hospital in Daegu, South Korea) from February 19, 2020 to March 26, 2020. The final date of follow-up was April 10, 2020.

During the study period, all consecutive adult patients (age ≥ 18 years) with SARS-CoV-2 infection admitted to the hospital via the emergency or outpatient department were eligible for inclusion. Seven patients who were transferred to other hospitals and whose final clinical results were unknown were excluded from the analyses.

Data collection and definitions

Patients' electronic medical records were reviewed. Data on patients' age, sex, comorbidities, vital signs, and complications were collected.

SIRS10 assesses the systemic inflammatory response to a variety of severe clinical situations. SIRS is manifested by two or more of the following conditions: temperature > 38°C or < 36°C; heart rate > 90 beats per minute; respiratory rate > 20 breaths per minute or PaCO2 < 32 mm Hg; and white-blood-cell count > 12,000/cu mm, < 4,000/cu mm, or > 10% immature (band) forms.

The qSOFA11 is a rapid bedside clinical scoring system for detecting patients with a suspected infection who are at higher risk for a poor clinical outcome. The qSOFA scale assigns a score of 1 point to each of three parameters related to organ function: respiratory rate, Glasgow coma scale, and systolic blood pressure. qSOFA scores of 2 or higher indicate higher in-hospital mortality.

The NEWS12 is an early warning score to improve early detection of, and response to, clinical deterioration. It is composed of seven parameters: pulse oximetry, oxygen, pulse rate, systolic blood pressure, respiration rate, temperature, and central nervous system status. Each parameter is assigned a score of 0–3 points. The score reflects the extent to which the parameters differ. NEWS is stratified into three categories: low risk (0–4), medium risk (5–6), and high risk (≥ 7).13

Critical outcomes were defined as ICU care or death. Critical patients were defined as patients representing the critical outcomes. Acute respiratory distress syndrome (ARDS) was defined according to the Berlin definition.14 Septic shock was defined according to the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3).15

Endpoints

Primary endpoints were the 28-day mortality and critical outcomes. Secondary endpoints were complications such as ARDS, septic shock, and ICU care.

Statistical analyses

Continuous variables are expressed as means ± standard deviations (SDs) and were compared using Student's t-test or the Mann-Whitney U test. Categorical variables were compared using the χ2 test or Fisher's exact test. Area under a receiver operating characteristic (AUROC) curve analyses were performed to assess the effectiveness of SIRS, qSOFA, and NEWS for predicting a COVID-19 prognosis. To compare the effectiveness of the three scoring systems, pairwise comparisons of AUROC curves were conducted. Survival probability was calculated according to the NEWS stratification using Kaplan-Meier analyses and compared using the log-rank test. In all analyses, P < 0.05 in two-tailed tests was considered to indicate statistical significance. All statistical procedures were performed using SPSS software (ver. 24.0; SPSS Inc., Chicago, IL, USA).

Ethics statement

This study was conducted in accordance with the tenets of the Declaration of Helsinki and was reviewed and approved by the Institutional Review Board of Yeungnam University Hospital (YUH IRB 2020–03–057). The requirement for informed consent was waived because of the retrospective study design.

RESULTS

Baseline characteristics and components of scoring systems in COVID-19 patients

The demographic and baseline characteristics of the patients are presented in Table 1. This study included 110 patients with SARS-CoV-2, 15 (13.6%) of whom were critical cases. Sixty two patients were women (56.4%) and their mean age was 56.9 ± 17.0 years. The critical group was significantly older than the non-critical group (mean age: 66.1 ± 10.0 vs. 55.4 ± 17.4 years, P = 0.002). The critical group was significantly more likely to have diabetes mellitus (66.7% vs. 20.0%, P < 0.001). Respiration rates were significantly higher in the critical group (24.3 ± 5.4 vs. 20.5 ± 1.6, P = 0.016). The critical group had many complications, such as ARDS (13/15 patients; 86.7%), septic shock (8/15; 53.3%), and ICU care (13/15; 86.7%), and 28-day mortality was high (6/15; 40.0%). SIRS, qSOFA, and NEWS differed significantly between the critical and non-critical groups.

Table 1

Baseline characteristics of the study population with COVID-19

Characteristics		All patients (n = 110)	Critical patients (n = 15)	Non-critical patients (n = 95)	P value
Age, yr		56.9 ± 17.0	66.1 ± 10.0	55.4 ± 17.4	0.002
Sex					0.577
	Male	48 (43.6)	8 (53.3)	40 (42.1)
	Female	62 (56.4)	7 (46.7)	55 (57.9)
Comorbidities		49 (44.5)	7 (46.7)	42 (44.2)	1.000
	Cardiovascular disease	10 (9.1)	0 (0)	10 (10.5)	0.352
	Cerebrovascular disease	4 (3.6)	1 (6.7)	3 (3.2)	0.449
	Chronic lung disease	4 (3.6)	1 (6.7)	3 (3.2)	0.449
	Dementia	4 (3.6)	1 (6.7)	3 (3.2)	0.449
	Diabetes mellitus	29 (26.4)	10 (66.7)	19 (20.0)	< 0.001
	Hypertension	37 (33.6)	6 (40.0)	31 (32.6)	0.770
	Malignancy	6 (5.5)	1 (6.7)	5 (5.3)	1.000
Vital signs on admission
	Body temperature, °C	37.2 ± 0.7	37.5 ± 0.8	37.1 ± 0.6	0.068
	Heart rate, beats/min	86.0 ± 13.8	90.5 ± 10.5	85.3 ± 14.1	0.174
	Respiration rate, breaths/min	21.0 ± 2.8	24.3 ± 5.4	20.5 ± 1.6	0.016
	Systolic BP, mmHg	128.1 ± 18.6	133.5 ± 19.2	127.3 ± 18.5	0.235
	Diastolic BP, mmHg	79.9 ± 12.2	82.5 ± 11.3	79.5 ± 12.4	0.388
	Mean arterial BP, mmHg	95.9 ± 12.8	99.5 ± 13.5	95.4 ± 12.6	0.252
Complications
	ARDS	18 (16.4)	13 (86.7)	5 (5.3)	< 0.001
	Septic shock	8 (7.3)	8 (53.3)	0 (0)	< 0.001
	ICU care	13 (11.8)	13 (86.7)	0 (0)	< 0.001
28-day hospital mortality		6 (5.5)	6 (40.0)	0 (0)	< 0.001
Scoring systems
	SIRS	0.9 ± 0.9	1.7 ± 1.0	0.8 ± 0.8	< 0.001
	qSOFA	0.3 ± 0.5	0.8 ± 0.6	0.2 ± 0.4	< 0.001
	NEWS	2.5 ± 3.0	7.3 ± 3.4	1.7 ± 2.1	< 0.001

Data are presented as the mean ± standard deviation or number (%).

ARDS = acute respiratory distress syndrome, BP = blood pressure, COVID-19 = coronavirus disease 2019, ICU = intensive care unit, NEWS = National Early Warning Score, qSOFA = quick Sepsis-related Organ Failure Assessment, SIRS = Systemic Inflammatory Response Syndrome.

Comparisons of components of the scoring systems in patients with COVID-19 are shown in Table 2. Based on SIRS, respiration-rate scores were significantly higher in the critical group (0.7 ± 0.5 vs. 0.2 ± 0.4, P < 0.001). Based on qSOFA, respiration-rate scores were significantly higher in the critical group (0.7 ± 0.5 vs. 0.1 ± 0.3, P < 0.001), while systolic BP scores were significantly lower in the critical group (0.0 ± 0.0 vs. 0.1 ± 0.3, P = 0.002). Based on NEWS, respiration rates, oxygen saturation, any supplemental oxygen, and heart-rate scores differed significantly between the two groups.

Table 2

Comparison of components of scoring systems in patients with COVID-19

Variables		All patients (n = 110)	Critical patients (n = 15)	Non-critical patients (n = 95)	P value
SIRS		0.9 ± 0.9	1.7 ± 1.0	0.8 ± 0.8	< 0.001
	Body temperature	0.1 ± 0.3	0.2 ± 0.4	0.1 ± 0.2	0.231
	Heart rate	0.3 ± 0.5	0.5 ± 0.5	0.3 ± 0.5	0.292
	Respiration rate	0.3 ± 0.4	0.7 ± 0.5	0.2 ± 0.4	< 0.001
	White blood cells with differentials	0.2 ± 0.4	0.3 ± 0.5	0.2 ± 0.4	0.492
qSOFA		0.3 ± 0.5	0.8 ± 0.6	0.2 ± 0.4	< 0.001
	Mental status	0.03 ± 0.2	0.1 ± 0.4	0.02 ± 0.1	0.242
	Respiration rate	0.2 ± 0.4	0.7 ± 0.5	0.1 ± 0.3	0.001
	Systolic BP	0.1 ± 0.3	0.0 ± 0.0	0.1 ± 0.3	0.002
NEWS		2.5 ± 3.0	7.3 ± 3.4	1.7 ± 2.1	< 0.001
	Respiration rate	0.4 ± 0.8	1.3 ± 1.3	0.2 ± 0.6	0.008
	Oxygen saturation	0.6 ± 1.1	2.2 ± 1.3	0.3 ± 0.8	< 0.001
	Any supplemental oxygen	0.5 ± 0.9	1.6 ± 0.8	0.3 ± 0.7	< 0.001
	Body temperature	0.1 ± 0.3	0.1 ± 0.3	0.1 ± 0.3	0.510
	Systolic BP	0.5 ± 0.8	0.8 ± 1.1	0.4 ± 0.8	0.087
	Heart rate	0.4 ± 0.6	1.0 ± 0.9	0.3 ± 0.4	0.008
	Mental status	0.1 ± 0.6	0.4 ± 1.1	0.1 ± 0.4	0.242

Data are presented as the mean ± standard deviation.

BP = blood pressure, NEWS = National Early Warning Score, qSOFA = quick Sepsis-related Organ Failure Assessment, SIRS = Systemic Inflammatory Response Syndrome.

Receiver operating characteristic (ROC) curves for the prediction of complications and mortality using the three scoring systems

Fig. 1 shows the ROC curves for the prediction of complications, 28-day mortality, and critical outcomes. The AUROCs for the identification of 28-day mortality were: SIRS = 0.639 (95% confidence interval [CI], 0.423–0.856), qSOFA = 0.779 (95% CI, 0.600–0.957), and NEWS = 0.867 (95% CI, 0.709–1.000). For predicting 28-day mortality, NEWS was superior to qSOFA (AUROC 0.867 vs. 0.779, P < 0.001), while there was no significant difference between NEWS and SIRS (AUROC 0.867 vs. 0.639, P = 0.100). With regards to critical outcomes, AUROC values were: SIRS = 0.744 (95% CI, 0.602–0.886), qSOFA = 0.760 (95% CI, 0.620–0.899), and NEWS = 0.918 (95% CI, 0.841–0.995). For predicting critical outcomes, NEWS was superior to both SIRS (AUROC, 0.918 vs. 0.744; P = 0.032) and qSOFA (AUROC, 0.918 vs. 0.760; P = 0.012). For predicting ARDS, NEWS was superior to both SIRS (AUROC, 0.928 vs. 0.720; P = 0.002) and qSOFA (AUROC, 0.928 vs. 0.760; P = 0.005). For predicting septic shock, NEWS was superior to both SIRS (AUROC, 0.952 vs. 0.719; P = 0.012) and qSOFA (AUROC, 0.952 vs. 0.740; P = 0.014). For predicting ICU care, NEWS was superior to qSOFA (AUROC, 0.937 vs. 0.776; P = 0.048), while there was no significant difference between NEWS and SIRS (AUROC, 0.937 vs. 0.794; P = 0.058).

Fig. 1

Comparison of SIRS, qSOFA, and NEWS for predicting clinical outcomes of COVID-19.

AUROC = Area under a receiver operating characteristic, ICU = intensive care unit, NEWS = National EarlyWarning Score, qSOFA = Quick Sepsis-related Organ Failure Assessment, SIRS = Systemic Inflammatory Response Syndrome.

Table 3 shows the prevalence of complications, 28-day mortality, and critical outcomes in different stratifications of the SIRS, qSOFA, and NEWS.

Table 3

Prevalence of complications, mortality, and critical outcomes in the different stratifications of SIRS, qSOFA and NEWS

Variables		ARDS	Septic shock	ICU care	28-day mortality	Critical outcomes
SIRS
	0–1	9 (50.0)	4 (50.0)	5 (38.5)	4 (66.7)	7 (46.7)
	≥ 2	9 (50.0)	4 (50.0)	8 (61.5)	2 (33.3)	8 (53.3)
qSOFA
	0–1	17 (94.4)	8 (100.0)	12 (92.3)	6 (100.0)	14 (93.3)
	≥ 2	1 (5.6)	0 (0)	1 (7.7)	0 (0.0)	1 (6.7)
NEWS
	0–4	3 (16.7)	0 (0)	1 (7.7)	1 (16.7)	2 (13.3)
	5–6	5 (27.8)	2 (25.0)	4 (30.8)	1 (16.7)	4 (26.7)
	≥ 7	10 (55.5)	6 (75.0)	8 (61.5)	4 (66.6)	9 (60.0)

Data are presented as number (%).

ARDS = acute respiratory distress syndrome, ICU = intensive care unit, NEWS = National Early Warning Score, qSOFA = quick Sepsis-related Organ Failure Assessment, SIRS = Systemic Inflammatory Response Syndrome.

Testing accuracy of NEWS according to risk stratification

The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of NEWS for prediction of critical outcomes were 86.7%, 90.5%, 59.1%, and 97.7%, respectively, when NEWS ≥ 5 (medium risk). With a threshold value of 7 points (high risk), these values were 60.0%, 96.8%, 75.0%, and 93.9%, respectively (Table 4).

Table 4

Sensitivity, specificity, PPV, and NPV of NEWS for prediction of critical outcomes

NEWS	Critical outcomes, %
	Sensitivity	Specificity	PPV	NPV
≥ 1	100.0	36.8	20.0	100.0
≥ 5	86.7	90.5	59.1	97.7
≥ 7	60.0	96.8	75.0	93.9

NEWS = National Early Warning Score, NPV = negative predictive value, PPV = positive predictive value.

Kaplan-Meier survival curves displaying 28-day mortality according to the different stratifications of NEWS

The Kaplan-Meier survival curves for cumulative risk of 28-day mortality according to the NEWS stratification are shown in Fig. 2. Survival time was significantly shorter for patients with NEWS ≥ 7 than for those with NEWS < 7 (log-rank test, P < 0.001).

Fig. 2

Kaplan Meier survival curves for cumulative risk of 28-day mortality according to the different stratifications of the NEWS.

NEWS = National Early Warning Score.

DISCUSSION

In this study, 110 hospitalised patients with SARS-CoV-2 infection were analysed, 15 (13.6%) of whom were critical cases, and the 28-day mortality rate was 5.5%. AUROC values for the prediction of 28-day mortality were: SIRS = 0.639 (95% CI, 0.423–0.856), qSOFA = 0.779 (95% CI, 0.600–0.957), and NEWS = 0.867 (95% CI, 0.709–1.000), respectively. With regards to the critical outcomes, AUROC values were: SIRS = 0.744 (95% CI, 0.602–0.886), qSOFA = 0.760 (95% CI, 0.620–0.899), and NEWS = 0.918 (95% CI, 0.841–0.995). The NEWS was effective for predicting complications of COVID-19 such as ARDS, septic shock, and ICU care. The NEWS ≥ 7 group had shorter survival times compared to the NEWS < 7 group. To the best of our knowledge, this is the first study to evaluate the effectiveness of NEWS for the prediction of mortality, critical outcomes, and complications in COVID-19 populations.

SIRS was widely used until the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) were issued in 2016.15 The previous definition of sepsis was infection with more than two of the four SIRS criteria focused on inflammatory excess, and SIRS criteria did not reflect dysregulated host responses or the presence of organ dysfunction from infection. If two of the four SIRS criteria were used to define sepsis, one in eight people with infection, organ failure, and significant mortality rates were excluded.16 In addition, almost 50% of patients hospitalised in general wards developed SIRS at least once during hospitalisation, suggesting that SIRS criteria have poor discriminant validity and are ineffective for detecting sepsis.17

Definitions of sepsis and septic shock were revised in 2016 by Sepsis-3.15 An increase in the SOFA score of more than two had better prognostic accuracy for predicting mortality compared to SIRS in patients with infection.51819 However, a SOFA score includes six parameters and requires laboratory tests (PaO2, platelets, bilirubin, and creatinine). Using simpler and more useful criteria, the qSOFA was introduced, which showed great prediction validity for in-hospital mortality (AUROC = 0.81; 95% CI, 0.80–0.82), and was statistically better than SOFA or a change in SOFA score in non-ICU patients.11 The NEWS showed a great ability to discriminate infected patients at high risk for clinical deterioration at the time of admission13 in the emergency department20 and outside the ICU.9

Our data focussed on COVID-19 patients at the time of admission. In our study, SIRS, qSOFA, and NEWS all had significantly high scores in the critical patient group. In most cases, there were significant differences in scores related to breathing and oxygenation. Sun et al.21 reported that oxygen supplementation is an independent risk factor for novel coronavirus pneumonia progressing to a critical condition, and blood oxygen saturation (SpO2) < 93% when breathing room air needs critical-care management. The Chinese Center for Disease Control and Prevention divided the clinical manifestations of the disease into three degrees of severity, and dyspnea, respiratory frequency ≥ 30/min, or an SpO2 ≤ 93% were classified as severe disease.2 Based on our results and other findings, difficulty in breathing, hypoxia, and any oxygen requirements in the early stage of hospitalisation are considered important indicators for predicting progress to a serious condition.

In our study, NEWS at admission showed equivalent or superior accuracy to that of SIRS and qSOFA for predicting clinical deterioration, such as ARDS, septic shock, ICU care, 28-day mortality, and critical outcomes in COVID-19 patients. As shown in Table 3, SIRS and qSOFA indicated a higher frequency of poor clinical outcomes in groups with lower risk. The low accuracy of SIRS and qSOFA prediction in COVID-19 clinical outcomes is that there are many “silent hypoxemia” patients in severe COVID-19. Patients with silent hypoxemia seem to breathe comfortably, but in fact, oxygen saturation if often low when measured through pulse oximetry. For this reason, SIRS and qSOFA has limitations in predicting the outcomes of COVID-19. The high accuracy of NEWS predictions in COVID-19 clinical outcomes is interpreted to be related to the aforementioned factors (breathing, hypoxia, and oxygen requirements), which are three of the variables in NEWS.

Early detection is important in COVID-19, as pneumonia caused by the novel coronavirus is often fatal due to rapid progression. In addition, no drug has been proven effective for COVID-19. Early interventions can be implemented for patients with more than five NEWS points, who are expected to develop disease progression. First, since COVID-19 is a highly contagious infectious disease, requiring medical staff to wear personal protective equipment, resulting in less contact with patients compared to other patients. Due to these limitations, the discovery of worsening patients may be delayed. NEWS allows intensive monitoring by selecting patients who are likely to progress with a critical illness. Second, the use of a high-flow nasal cannula (HFNC) or noninvasive ventilation (NIV) can be considered in patients with ARDS with alveolar collapse or hypercapnia. By early identification of high-risk patients, these non-invasive measures can give patients a chance to recover. Third, due to the nature of the pandemic, medical personnel, facilities, and equipment are often depleted. NEWS can help to detect high-risk patients quickly and early, and thus save patients by emergency evacuation to locations where medical personnel, facilities and equipment are available.

This study had several limitations. First, because it was a retrospective study conducted at a single centre with a relatively small number of COVID-19 patients, the results cannot be generalised. External validation is needed with studies on future large cohorts of COVID-19 patients to confirm the efficacy of NEWS for predicting poor clinical outcomes. Second, the efficacy of the former SOFA was not assessed. Because contact with COVID-19 patients is challenging, obtaining arterial blood gas analyses for measuring PaO2/FiO2 ratios was not feasible for some patients. Third, selection bias cannot be avoided. We did not use data from a large population and the severity of COVID-19 may differ among hospitals and between countries. However, a strength of our study is that it was the first to evaluate the efficacy of preexisting scoring systems for the prediction of clinical deterioration in COVID-19 patients.

In conclusion, calculation of the NEWS at the time of hospital admission can predict critical clinical outcomes in patients with COVID-19, and its predictive value is superior to that of SIRS and qSOFA. By early detection of the high-risk group using NEWS, early interventions for high-risk patients can improve clinical outcomes in COVID-19 patients.