Predictive Value of Sequential Organ Failure Assessment, Quick Sequential Organ Failure Assessment, Acute Physiology and Chronic Health Evaluation II, and New Early Warning Signs Scores Estimate Mortality of COVID-19 Patients Requiring Intensive Care Unit.

Introduction: Various mortality predictive score models for coronavirus disease-2019 (COVID-19) have been deliberated. We studied how sequential organ failure assessment (SOFA), quick sequential organ failure assessment (qSOFA), acute physiology and chronic health evaluation II (APACHE II), and new early warning signs (NEWS-2) scores estimate mortality in COVID-19 patients. Materials and methods: We conducted a prospective cohort study of 53 patients with moderate-to-severe COVID-19. We calculated qSOFA, SOFA, APACHE II, and NEWS-2 on initial admission and re-evaluated on day 5. We performed logistic regression analysis to differentiate the predictors of qSOFA, SOFA, APACHE II, and NEWS-2 scores on mortality. Result: qSOFA, SOFA, APACHE II, and NEWS-2 scores on day 5 exhibited a difference between survivors and nonsurvivors (p <0.05), also between ICU and non-ICU admission (p <0.05). The initial NEWS-2 revealed a higher AUC value than the qSOFA, APACHE II, and SOFA score in estimating mortality (0.867; 0.83; 0.822; 0.794). In ICU, APACHE II score revealed a higher AUC value than the SOFA, NEWS-2, and qSOFA score (0.853; 0.832; 0.813; 0.809). Concurrently, evaluation on day 5 showed that qSOFA AUC had higher scores than the NEWS-2, APACHE II, and SOFA (0.979; 0.965; 0.939; 0.933) in predicting mortality, while SOFA and APACHE II AUC were higher in ICU admission than NEWS-2 and qSOFA (0.968; 0.964; 0.939; 0.934). According to the cutoff score, APACHE II on day 5 revealed the highest sensitivity and specificity in predicting the mortality (sensitivity 95.7%, specificity 86.7%).
Conclusion: All scores signify good predictive values on COVID-19 patients mortality following the evaluation on the day 5. Nonetheless, APACHE-II appears to be the best at predicting mortality and ICU admission rate. How to cite this article: Asmarawati TP, Suryantoro SD, Rosyid AN, Marfiani E, Windradi C, Mahdi BA, et al. Predictive Value of Sequential Organ Failure Assessment, Quick Sequential Organ Failure Assessment, Acute Physiology and Chronic Health Evaluation II, and New Early Warning Signs Scores Estimate Mortality of COVID-19 Patients Requiring Intensive Care Unit. Indian J Crit Care Med 2022;26(4):464-471.

Introduction

December 2019 was formerly a period when severe acute respiratory syndrome coronavirus-2 (SARS CoV-2) emanated and caused an infectious disease named coronavirus disease-2019 (COVID-19), which began expanding immensely in numbers in Wuhan, China.[1-3] However, this pandemic is still way complex and uncontrolled, as evidenced by the new wave that triggered the rise of new cases so that several countries got to be surely prepared in handling COVID-19 patients. The government took step began opening of emergency hospitals, requesting for medical volunteers, and providing facilities that support the management of COVID-19 patients.[3]

Unfortunately, this pandemic has not yet reached an end as the virus transmission continues. Several studies have attempted to examine predictors of mortality from clinical, risk factors, and laboratories in order to assess outcomes in COVID-19 patients. On the other hand, some severe COVID-19 patients treated in the intensive or high care unit (ICU or HCU) provided satisfactory outcomes. Clinical assessment of disease severity is necessary for determining intervention, severity, outcome prediction, and prognosis.[4,5]

Diverse models of mortality prediction referring to disease severity have been initiated in patients with multiple diagnoses. Information derived from this predictor model can be used to evaluate hospitalization in the ICU, improve emergency interventions, and allocate resources appropriately.[4] Regarding the condition of COVID-19 patients that changes instantly, a scoring model that can be evaluated easily on the bed-side and followed up from time to time makes it easier for clinicians to convey the condition to the patient's family.[6-8]

Several studies attempted to map the prognosis of critically ill COVID-19 patients using scores, such as sequential organ failure assessment (SOFA), quick sequential organ failure assessment (qSOFA), and new early warning signs (NEWS-2).[6,9-13] Acute physiology and chronic health evaluation II (APACHE II) itself has been involved in several studies aimed at evaluating critically ill patients requiring ventilatory care.[14-16] These scores are intended to evaluate the complication sequence of the disease causing the morbidity, not to predict the outcome. Nevertheless, there is close relationship between organ failure and survival patient.

We calculated these four scores and followed them to define their relationship to mortality of COVID-19 patients requiring ICU care.

Materials and Methods

We conducted a prospective cohort study of moderate to severe COVID-19 patients hospitalized to Universitas Airlangga Hospital. We calculated the sample size for the predictive study and got a sample of 53 COVID-19 patients of moderate-to-severe grade. We used the calculation of the minimum number of samples using the cross-sectional method with a sample size of 50. We used the consecutive sampling method and followed the patients from the beginning of admission to the day of discharge. We analyze qSOFA, SOFA, NEWS-2, and APACHE II scores at initial admission (baseline) and on day 5 during hospitalization. We calculate qSOFA by identifying and including respiratory rate, Glasgow coma scale, and systolic blood pressure. Furthermore, SOFA score quantifies PaO2/FiO2 ratio, platelets, total bilirubin, mean arterial pressure, Glasgow coma scale, creatinine, and urine output per day. Another score, named NEWS-2, measures respiratory rate, oxygen saturation (SaO2), oxygen support, systolic blood pressure, pulse, consciousness, and temperature. APACHE-II assesses patients temperature, mean arterial pressure, heart rate, respiratory rate, oxygenation, arterial pH, serum sodium, serum potassium, serum creatinine, hematocrit, white blood count, Glasgow coma scale, serum bicarbonate (HCO3). The main outcome of this study was mortality and ICU-admission in COVID-19 patients. Multivariate analysis through logistic regression was carried out to obtain the predictor value of the four scores on mortality.

Statistical Analysis

We analyzed the data using SPSS version 24 (Chicago, Illinois, USA; RRID: SCR_002865), which is an open-access alternative. Baseline characteristic subjects (Table 1) are described as mean ± standard deviation or median number. We analyzed predictors of mortality with univariate and multivariate cox regression analysis. Mortality threshold from each score of SOFA, qSOFA, NEWS-2, and APACHE II was performed; the receiver–operating curve (AUC) analysis associated with the area under the curve (AUC) was used to analyze optimal parameter value of the laboratory to predict the progression of mortality in the study group. Excellent AUC lies between 0.9 and 1; good if 0.8

Table 1

Baseline characteristic of patients study

Sex	n		%
Male (n, %)	27		49.1
Female (n, %)	26		50.9
Age (mean/SD)	53.49		12.49
Duration hospitalization	15.76		7.52
Severity of COVID-19
Moderate	13		24.5
Severe	11		20.8
Critically ill	29		54.7
Comorbid
DM (n, %)	28		52.8
HT (n, %)	23		43.4
Heart disease (n, %)	6		11.3
Tuberculosis (n, %)	5		9.4
Kidney disease (n, %)	3		5.7
Liver disease (n, %)	2		3.8
Critical status
ARDS (n, %)	27		50.9
Respiratory failure (n, %)	32		60.4
Sepsis (n, %)	14		26.4
Shock sepsis (n, %)	10		18.9
Ventilator (n, %)	25		47.2
Outcome
ICU admission (n, %)	26		49.1
Death (n, %)	23		43.4
Vital sign	Day 0		Day 5
Systolic blood pressure (mm Hg)	119.08	22.78	104.55	30.53
Diastolic blood pressure (mm Hg)	74.74	15.79	63.68	17.12
Heart rate (x/minute)	106.77	15.85	110.23	21.82
Respiration rate (x/minute)	23.58	3.89	21.89	2.87
Temperature (°C)	36.6	0.59	36.75	0.68
Saturation oxygen (%)	94.99	19.65	96.65	22.85
Laboratory (median, SD)	Day 0		Day 5
Hb (g/dL)	12.9	1.97	12.7	2.1
Hct	37.8	5.44	37	6.67
WBC (103/µL)	9.8	6.46	10.14	7.25
Nuetrophil (%)	82	18.49	84	14.88
Lymphocyte (%)	9.9	12.99	9	11.37
Absolute lymphocyte count (/µL)	1426	1860.27	1100	1641.29
NLR	8.38	13.4	9	13.98
CRP (mg/L)	21.14	53.59	34.12	42.38
Procalcitonin (ng/mL)	0.86	3.82	0.58	2.03
BUN (mg/dL)	15.5	30.51	20.4	25.34
Creatinine serum (mg/dL)	0.88	2.32	0.9	1.98
AST (µ/L)	44	354.56	50	152.71
ALT (µ/L)	43	134.39	56	97.16
Albumin (g/dL)	3.48	0.53	3.4	0.75
Total bilirubin (mg/dL)	0.57	3.69	0.6	5.8
Electrolyte serum
Natrium (mmol/L)	137	4.95	137	5.79
Potassium (mmol/L)	3.9	5.15	4	0.89
Chloride (mmol/L)	104	5.65	103	4.87
Blood gas analysis
pCO2 level (mm Hg)	32.6	12.6	41.9	18.11
HCO3 level (mmol/L)	20	5.02	22.7	7.23
PF ratio	155.1	90.88
Scoring (mean; SD)
qSOFA	1.38	0.94	1.64	1.18
SOFA	4.98	3.08	6.34	4.23
APACHE II	12.19	7.5	14.6	9.52
NEWS-2	9.58	3.05	10.57	4.11

APACHE II, acute physiology and chronic health evaluation II; ARDS, acute respiratory distress syndrome; ALT, alanine transaminase; AST, aspartate transaminase; BUN, blood urea nitrogen; CRP, C-reactive protein; DM, diabetes mellitus; HT, hypertension; ICU, intensive care unit; NLR, neutrophil-lymphocyte ratio; NEWS-2, new early warning signs; qSOFA, quick sequential organ failure assessment; SOFA, sequential organ failure assessment; WBC, white blood cell

Result

Fifty-three eligible patients were included in the study. There were 27 male patients (50.9%) with a mean age of 53.49 ± 12.49 years. The study population required ICU with 29 (54.7%) patients were in critical illness. Diabetes mellitus (52.8%) and hypertension were the most frequent comorbidities (43.4%). In general, patients required ICU treatment due to respiratory failure (32; 60.4%), acute respiratory stress disorders (ARDS) (27; 50.9%), and the requirement of a ventilator (25; 47.2%) (Table 1). The average SOFA score for nonsurvivors (6.74 and 10.09) was higher than that for survivors (p <0.05). All predictive scores in this study, which are qSOFA, SOFA, NEWS-2, as well as APACHE II, statistically demonstrated a mean difference observed on the initial day and day 5 of hospitalization, and comparative data between survivor models are presented in Table 2. The goodness of t Hosmer-Lemeshow test and p-value is shown in Table 3. The overall discriminative capability as determined by the AUC curve is shown in Figure 1. The initial NEWS-2 revealed a higher AUC value than the qSOFA, APACHE II, and SOFA score in estimating mortality (0.867; 0.83; 0.822; 0.794). Admission to ICU, APACHE II score revealed a higher AUC value than the SOFA, NEWS-2, and qSOFA score (0.853; 0.832; 0.813; 0.809). Concurrently, according to the evaluation of the scores on day 5, qSOFA and NEWS-2 AUC had higher scores in predicting mortality than the APACHE II and SOFA (0.979; 0.965; 0.939; 0.933), while SOFA and APACHE II AUC showed higher values in ICU admission than NEWS-2 and qSOFA (0.968; 0.964; 0.939; 0.934).

Table 2

Univariate analysis of the scoring system

Scoring model	Survivors		Nonsurvivors		p value	Non-ICU admission		ICU admission		p value
	1 *	2 **	1	2		1	2	1	2
qSOFA	0.9	0.8	2	2.74	<0.05	0.89	0.74	1.88	2.58	<0.05
SOFA	3.63	3.47	6.74	10.09	<0.05	3.3	2.85	6.73	9.96	<0.05
APACHE II	8.73	7.97	16.7	23.26	<0.05	7.93	6.93	16.62	22.58	<0.05
NEWS-2	7.87	7.57	11.83	14.48	<0.05	8	7.48	11.23	13.77	<0.05

*model 1, taken on the day of admission; **model 2, taken on day 5 of hospitalization; APACHE II, acute physiology and chronic health evaluation II; NEWS-2, new early warning signs; qSOFA, quick sequential organ failure assessment; SOFA, sequential organ failure assessment

Table 3

Comparison of survivors vs nonsurvivors COVID-19 patients

Scoring model	Chi-square	p value
qSOFA1*	1.316	0.518
qSOFA2**	1.857	0.395
SOFA 1	1.404	0.966
SOFA 2	7.166	0.412
APACHE II (1)	4.611	0.798
APACHE II (2)	11.863	0.157
NEWS-2 (1)	3.181	0.786
NEWS-2 (2)	9.522	0.3

*(1), taken on the day of admission; **(2), taken on day 5 of hospitalization; APACHE II, acute physiology and chronic health evaluation II; NEWS-2, new early warning signs; qSOFA, quick sequential organ failure assessment; SOFA, sequential organ failure assessment

Fig. 1A

Area under the curve (AUC), sequential organ failure assessment (SOFA), quick sequential organ failure assessment (qSOFA), acute physiology and chronic health evaluation II (APACHE II), and new early warning signs (NEWS-2) score with mortality in COVID-19 patients

Area under the curve (AUC), sequential organ failure assessment (SOFA), quick sequential organ failure assessment (qSOFA), acute physiology and chronic health evaluation II (APACHE II), and new early warning signs (NEWS-2) score with ICU admission in COVID-19 patients

Referring to the cutoff analysis, it appeared that the four scores had poor sensitivity and specificity in the initial evaluation. However, on day 5, the sensitivity and specificity of the four scores showed high values. APACHE II had the highest sensitivity and specificity in predicting mortality (sensitivity 95.7% and specificity 86.7%) compared to others. Another four scores had poor sensitivity and specificity in estimating ICU admission. Furthermore, that NEWS-2 had the highest sensitivity with low specificity (sensitivity 96.4% and specificity 7.4%) (Fig. 1).

Discussion

Several previous publications regarding predictor models of COVID-19 patient outcome have involved clinical, comorbid, and laboratory factors. Only a few have focused on the predictive value of scores in severe COVID-19 patients.[17] The ideal score for assessing organ dysfunction needs to be objective, easy, uncomplicated, able to be evaluated regularly, organ-specific, having continuous variables, and independent.[7] Evaluation of these scores is an attempt to improve the quality of observation and therapy in treating COVID-19 patients in high and intensive care units.

SOFA was formed by consensus in 1994. The SOFA score is actually intended to determine the morbidity sequence of critically ill patients. SOFA score is usually evaluated when the patient is admitted to the ICU and every 24 hours during the treatment period.[18]

SOFA is a scoring system with high accuracy and has been widely practiced as a tool to identify a patient's organ function status.[19] It should be noted that SOFA is not designed to predict mortality but to describe the sequence of complications of organ dysfunction in critically ill patients. Even though subsequent assessment has an impact on its relationship to mortality, SOFA is intended to assess patient morbidity. Some of the parameters that are routinely performed in the ICU setting may not be replaced by these scores, but SOFA or APACHE II is able to complete their function because patient's mortality rate tends to be related to the degree of organ failure, which seems relevant if the mortality outcome is related to the SOFA score.[7]

SOFA score contains parameters of respiration (PaO2/FiO2), coagulation (platelet), liver (bilirubin), cardiovascular (hypotension), central nervous system, and renal function. In our analysis, we showed that the SOFA model can predict in-hospital mortality of COVID-19 patients in the ICU. At the time of hospitalization, SOFA score <3 became a predictor for the survival of COVID-19 patients. The advantage of this SOFA score included the use of six parameters that were always evaluated during treatment. Research by Elhadi et al. and Liu et al. showed the same finding that survivors scored <3.[19-21] Some SOFA components have also been the predictors of death in COVID-19 patients, such as low platelet count, increased serum creatinine, and bilirubin, as stated by Du.[3]

Recognition of sepsis is the cornerstone of therapy for severe infections. As previously written, SOFA is a score that can be used to predict and detect the occurrence of sepsis and risk stratification for critically ill patients. qSOFA is a score that was introduced to identify septic patients outside the ICU. However, qSOFA itself has a crucial practical parameter for the evaluation of critically ill patients requiring immediate intensive care.[22,23]

The qSOFA score was originally established by Seymour et al. who analyzed 148,907 patients with suspected infection in both the ICU and non-ICU. The qSOFA score is a simple score consisting of three items: respiratory rate, altered mental status, and systolic blood pressure. A qSOFA ≥2 was found to be significantly predictive of increased mortality in patients outside of the ICU.[23] A meta-analysis involving 229,480 infected patients showed that qSOFA score performed well for predicting mortality. In addition, another study by Kovach et al. analyzed that qSOFA showed AUC >0.8.[13]

The study by Liu et al. comparing SOFA and qSOFA in COVID-19 patients showed that the subgroup of persons aged <65 years were 0.912 and 0.703, and the difference between them was no statistically significant (p = 0.316). The AUC of SOFA and qSOFA scores in the subgroup of persons aged 65 years were 0.921 and 0.773, and the difference between those two was found to be statistically significant (p = 0.033). This study shows that the total qSOFA ≥2 in critical COVID-19 patients is associated with a high mortality rate.[13]

From the research above, it was found that the accuracy of the area under curve from qSOFA is lower than the SOFA score, so it can be concluded that SOFA is more accurate. The advantage of using SOFA is that it involves more complete parameters than qSOFA. However, it cannot be ruled out that the qSOFA is a score that is easy, simple, and fast to use in patient settings in the emergency room.[13] Our study showed that the predictive value for the qSOFA score revealed a higher AUC value than the SOFA score in predicting mortality. Performance score in initial evaluation showed AUC predictor on mortality qSOFA 0.83; SOFA 0.794 with a consistent evaluation on day 5 of treatment, the qSOFA had best result for predicting mortality (AUC 0.979). This data also presented the most similar result from previous research which was a cutoff value of qSOFA ≥2, while SOFA ≥5 indicated an increased risk of death in COVID-19 patients. Thus, the qSOFA score is able to appear as a practically quick scoring system to evaluate the deteriorating condition of critically ill patients.

NEWS-2 is an updated score from the NEWS score in 2017. NEWS-2 consists of six physiological parameters that can be obtained during patient visits. It is related to respiratory component, such as oxygen saturation and oxygen supplementation. Ideally, NEWS-2 is used in patients with ICU care and evaluated every 24 hours during treatment. In several studies, a score >5 NEWS-2 indicates an urgent response while a score of 7 or more triggers a clinical alert. The NEWS-2 score itself has the advantage of being able to represent hypoxia and oxygen supplementation. Therefore, NEWS-2 may perform better than the qSOFA score.[10-12]

A meta-analysis by Zhang examining the potential of the NEWS-2 score to predict clinical distress in COVID-19 patients showed good sensitivity and specificity were 0.75 (95% CI: 0.63, 0.84) and 0.65 (95% CI: 0.52, 0.76). This study also showed that threshold 5 shows good sensitivity (0.83), moderate specificity (0.65), and good discrimination (0.82).[12] So the score >5 is a warning for the paramedic team to immediately make the best intervention for COVID-19 patients.

In a Chinese study of 654 COVID-19 admissions, baseline NEWS2 predicted mortality with a AUC of 0.81 (95% CI 0.77–0.85). In a Norwegian study of 66 inpatients, baseline NEWS2 predicted a composite adverse outcome of inpatient mortality and/or ICU admission with AUC of 0.79 (95% CI 0.66–0.91) better than qSOFA 0.62 (0.45–0.81).[9] In our study, the NEWS-2 score performed best prediction for mortality (0.867) with cutoff ≥5.

APACHE II score is the sum of the components of physiology, age, and chronic conditions. APACHE II has become an accurate measure of the relationship between critical patient outcomes. This score has shown effectiveness, efficacy, and quality in each patient. In one study, the APACHE II score showed a general mortality predictor in the ICU with an area under the curve (AUC) from 0.74 to 0.86.[24]

The APACHE II score is one of several ICU scores that functions to measure disease severity and describe patient morbidity and prognosis. The APACHE II score helps predict mortality by estimating disease severity. APACHE II components include 13 physiological variables and 4 disease history variables. The APACHE II assessment began at the time admission with a maximum score of 71. A score of 25 predicted 50% mortality, while 35 predicted 80% mortality.[14] In our study, the APACHE II score had an under-curve area of 0.939 with p <0.05. Similar to the study conducted by Sam et al., the APACHE II score appeared to be linearly correlated with outcome (r = 0.347).[14] From this study, APACHE II ≥7 increased mortality risk and better predicted ICU admission with higher AUC value than the SOFA, qSOFA and NEWS-2 score (0.853; 0.832; 0.809; 0.813).

Furthermore, the study showed that the SOFA score is a highly sensitive marker of in-hospital mortality in COVID-19 patients and is prognostically superior to qSOFA in this setting. Nevertheless, there is a need to recognize that, as qSOFA is simple, fast, and acceptable accuracy, it can be used in the emergency room or at admission if the parameters of the SOFA cannot be acquired in time. We used the consecutive sampling method with an adequate number of samples so that this makes the internal validity sufficient. Its method was randomized by itself. Although the therapy and development of COVID-19 science have developed rapidly in a short time, the application of this score is expected to help stratify the risk of patients being treated. Therefore, considering that the COVID-19 wave has reached several peaks, it may be possible to find differences in the sensitivity of this score in some areas.

Our research was only carried out in a single-center hospital so that it may show distinct results due to population characteristics. There may be differences in the range of qSOFA, SOFA, NEWS-2 and APACHE II scores in assessing mortality in COVID-19 patients. We also did not include any evaluation of treatment options in COVID-19 patients, in regards to the ongoing development of COVID-19 treatment modalities, which might be a confounding factor.

Conclusion

SOFA, qSOFA, NEWS-2, and APACHE II scores seemed to finely predict the COVID-19 patients’ mortality at the ICU after the fifth day. Nonetheless, APACHE-II appears to be the best at predicting both mortality and ICU admission rate. At this point, taking further analysis comparing these four scorings into consideration is substantial.

Orcid

Tri Pudy Asmarawati https://orcid.org/0000-0001-9869-5098

Satriyo Dwi Suryantoro https://orcid.org/0000-0002-0522-8659

Alfian Nur Rosyid https://orcid.org/0000-0001-7042-996X

Erika Marfiani https://orcid.org/0000-0002-1918-0414

Choirina Windradi https://orcid.org/0000-0002-3394-1400

Bagus Aulia Mahdi https://orcid.org/0000-0001-6634-6088

Heri Sutanto https://orcid.org/0000-0003-4559-1839