Diagnostic and Prognostic Utility Compared Among Different Sepsis Scoring Systems in Adult Patients With Sepsis in Thailand: A Prospective Cohort Study.

BACKGROUND: The diagnostic and prognostic utility of various sepsis scores varied among different cohorts and settings.
METHODS: A prospective cohort study in adult patients with sepsis at Siriraj Hospital (Bangkok, Thailand) was conducted during January to July 2019. The performance of sepsis assessments, including systemic inflammatory response syndrome (SIRS) score, sequential organ failure assessment (SOFA) score, quick sepsis-related organ failure assessment (qSOFA) score, modified early warning score (MEWS), and national early warning score (NEWS), for sepsis detection and mortality prediction were compared with agreement between 2 infectious disease (ID) specialists to determine their sepsis and septic shock status as the reference standard.
RESULTS: Among the 470 subjects included in this study, 206 patients (43.8%) were determined by 2 ID specialists to have sepsis. Systemic inflammatory response syndrome ≥2, qSOFA ≥2, and NEWS ≥5 yielded the highest sensitivity (93.2%), specificity (81.3%), and accuracy (72.6%), respectively, for detecting sepsis. The SIRS ≥2 had the highest sensitivity (97.8%), whereas qSOFA ≥2 had the highest specificity (61%) and accuracy (69.7%) for predicting mortality among sepsis patients. Receiver operating characteristic (ROC) curve showed MEWS to have the highest discriminatory power for sepsis detection (area under the ROC curve [AUROC], 0.79; 95% confidence interval [CI], 0.74-0.83), whereas SOFA had the highest discriminatory power for predicting hospital mortality (AUROC, 0.76; 95% CI, 0.69-0.79).
CONCLUSIONS: The NEWS ≥5 and qSOFA ≥2 were the most accurate scoring systems for sepsis detection and mortality prediction, respectively. Each scoring system is useful for different specific purposes relative to early detection and mortality prediction in sepsis patients.

Sepsis is a life-threatening condition characterized by multiple organ dysfunction that is caused by an overwhelming host response to infection [1]. Sepsis remains a major public health problem worldwide with an estimated 11 million sepsis-related deaths in 2017 [2]. According to data from the Ministry of Public Health and National Health Security Office of Thailand, approximately 175 000 patients develop sepsis each year, and approximately 45 000 of those patients die. In Thailand, the mortality rate among patients with community-acquired sepsis was approximately 32% in 2017 [3]. There is substantial variation in sepsis incidence and mortality across regions, with the highest burden in low- to middle-income countries (LMICs) [2, 4]. Sepsis-specific epidemiological data in the Southeast Asian population are still limited [5]. Early recognition of sepsis can improve outcome via timely and appropriate interventions [6]. However, no gold standard criteria for detecting patients with sepsis have been established.

The systemic inflammatory response syndrome (SIRS) criteria have been used as a screening tool for the diagnosis of sepsis since 1991. However, due to the inadequate sensitivity and specificity of the SIRS criteria [7], the sequential organ failure assessment (SOFA) score and the quick sepsis-related organ failure assessment (qSOFA) score were proposed as new diagnostic criteria for sepsis by the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) in 2016. Performance of the qSOFA score for prediction of mortality in sepsis patients outside the intensive care unit (ICU) was reported from many studies in high-income countries [8-10]. An alternative scoring system for diagnosis of sepsis, the early warning score (EWS), was developed to meet the needs of different types of patients, such as the pediatric early warning score (PEWS), the national early warning score (NEWS), and the modified early warning score (MEWS). A comparative study reported that NEWS had greater prognostic accuracy than qSOFA or SIRS for hospital mortality, ICU transfer, and ICU length of stay in patients with sepsis [11]. However, there are limited data on the utility of the qSOFA score in LMICs [12].

The predictive validity of different sepsis scoring systems varies among patient cohorts and settings. Moreover, no studies have comprehensively evaluated and compared the different sepsis detection scoring systems in the Thai population. Accordingly, this study aimed to determine the diagnostic performance of SIRS score, qSOFA score, SOFA score, MEWS, and NEWS for sepsis detection and mortality prediction in adult patients suspected of having sepsis at Siriraj Hospital, Mahidol University, Bangkok, Thailand.

METHODS

Study Design and Population

This prospective observational study was conducted at Siriraj Hospital, which is the largest university-based national referral center in Thailand, during January 2019 to July 2019. Eligible patients were hospitalized adults aged 18 years or older who had blood cultures performed.

Patient Consent Statement

The protocol for this study was approved by the Institutional Review Boards of Siriraj Hospital (COA no. SI 597/2019), and written informed consent was waived because the research involves no more than minimal risk to the subjects.

Study Procedure

A list of hospitalized patients from all wards, including the ICU, who had blood culture performed was compiled, and patients were randomly selected using the random generator feature of R program software. A list of randomly selected patients was then provided to the research physician twice weekly. Approximately 100 to 120 subjects were recruited monthly. Clinical and laboratory data relating to SIRS score, qSOFA score, SOFA score, MEWS, and NEWS were collected at the time of blood draw for culture or within 6 hours before blood culture. The worst values for each item of the aforementioned scoring systems were used. Each patient was included only once on the date of suspected infection with bacteremia or sepsis or septic shock. The follow-up blood culture was not taken into account for determining diagnostic and prognostic utility compared among the different sepsis scoring systems. Patient clinical and laboratory data for each sepsis scoring system are shown in Supplementary Table S1. Presence of infection was determined by attending physicians at the time blood cultures were taken. The following data were also obtained from medical records: comorbidity, site and type of infection, causative pathogen, antimicrobial therapy, fluid and vasopressor therapy, and mortality.

Presence of sepsis and septic shock was determined by the opinion of 2 infectious disease (ID) specialists who reviewed the clinical, microbiological, and radiological data of each enrolled study subject, as well as the results of other diagnostic tests performed in the hospital until the patient died or was discharged from the hospital. Each of the 2 ID specialists independently reviewed the aforementioned data of each enrolled study subject to determine whether that patient had sepsis/septic shock. During that process, neither ID specialist knew the opinion of the other ID specialist. Any disagreement between the 2 ID specialists was decided by the opinion of a third ID specialist. In brief, the process involved first determining the presence of a particular kind of infection. The second criterion that had to be met to define sepsis was the presence of dysfunction in at least 1 organ/system, including cardiovascular, hematological, liver, neurological, respiratory, or renal involvement.

Definitions

Sepsis was defined as the presence of infection with organ dysfunction based on the clinical features, biochemical laboratory test results, microbiological findings, and the clinical course of the subject. Septic shock was defined as sepsis with persisting hypotension requiring vasopressor to maintain a mean arterial pressure of 65 mmHg or having a serum lactate level >2 mmol/L despite adequate volume resuscitation [8]. In our study, we classified infections into 3 groups according to the setting where the infection was acquired. Community-acquired infection was defined as an infection in a patient who was hospitalized ≤2 days, who had no healthcare-associated conditions, and who was not hospitalized in other hospitals longer than 2 days before transfer to this hospitalization. Hospital-acquired infection was defined as an infection in a patient who was hospitalized >2 days or hospitalized in other hospitals longer than 2 days before transfer to this hospitalization. Healthcare-associated infection was defined as an infection in a patient who was hospitalized ≤2 days that had a history of prior hospitalization or prior use of antibiotic within the preceding 90 days, who was a resident in a nursing home or extended care facility, or who had chronic dialysis within 30 days. Concordant empiric antibiotic therapy (ABT) was defined as the identified organism being susceptible to the given empiric ABT. Definite ABT was defined as the ABT given subsequently according to the result of susceptibility testing.

Outcomes

The primary outcome was the performance and accuracy of different sepsis scoring systems for detecting sepsis or septic shock compared with agreement in the opinions of 2 ID specialists as the gold standard. The secondary outcomes included (1) performance and accuracy of each scoring system for predicting hospital mortality in sepsis patients and (2) identification of factors independently associated with mortality in sepsis patients.

Sample Size Estimation

Based on the results of a previous study, we estimated the sensitivity of qSOFA for detecting sepsis to be 80% ± 10%. Using a type I error (2-sided) of 5%, the estimated number of sepsis patients was 62. That number was increased by 10% to compensate for missing data, which increased our minimum sample size to approximately 70 patients.

Information from the Global Antimicrobial Resistance Surveillance System (GLASS) at Siriraj Hospital in 2016 showed that the number of hospitalized patients who have blood cultures performed at Siriraj Hospital is approximately 5000 patients per year. The rate of sepsis in patients with positive blood culture specimens was approximately 15% [13]. Therefore, 467 adult subjects who had blood culture performed was estimated to yield at least 70 sepsis subjects and approximately 397 nonsepsis subjects.

Statistical Analysis

Characteristics of enrolled subjects were described using descriptive statistics. Continuous variables are reported as mean ± standard deviation for normally distributed variables and as median (25th percentile and 75th percentile) or range for nonnormally distributed variables. Student’s t test or Mann-Whiney U test was used for comparison of continuous variables, whereas χ 2 test or Fisher’s exact test was used for comparison of categorical variables. All statistical tests were 2 sided, with a P value less than .05 indicating statistical significance. Variables with a P < .05 were further analyzed for independent association with mortality using binary logistic regression analysis with a forward stepwise method. Those results are presented as odds ratio (OR) and 95% confidence interval (CI). Performance measures, including sensitivity, specificity, negative and positive predictive values, and accuracy, of various scoring systems for sepsis detection and mortality prediction were calculated as percentage and 95% CI. We evaluated the accuracy of various scoring systems for sepsis detection compared with agreement in the opinions of 2 ID specialists, which was considered the reference standard for sepsis and septic shock. Accuracy was calculated as the percentage of correctly classified instances (true positive + true negative)/(true positive + true negative + false positive + false negative). Diagnostic and prognostic accuracy of the various scoring systems were analyzed and compared by receiver operating characteristic (ROC) curve and area under the ROC curve (AUROC) values with 95% CI. All analyses were performed using PASW Statistics (SPSS) 18.0 (SPSS, Inc., Chicago, IL).

RESULTS

Among the 558 adult subjects with blood culture taken during the study period, 88 subjects were duplicate subjects who had already been previously recruited into the study. Among the 470 subjects included in this study, 409 were diagnosed with infection. Of those, 206 and 203 subjects were determined by 2 ID specialists to have and not have sepsis, respectively, for an incidence of sepsis of 43.8% among patients who had blood cultures performed. The concordance of interobserver agreement in the judgment of sepsis and septic shock was 78.6% (95% CI, 74.5%–82.2%) and 91.3% (95% CI, 88.3%–93.7%), respectively.

The baseline characteristics of subjects with and without sepsis are shown in Table 1. Two hundred thirty-seven (57.9%) patients had a positive culture result from any site, and 117 patients (28.6%) had a positive culture from blood, which was defined as true bacteremia. Thirty-one positive blood cultures were considered contaminated (20 in sepsis [9.7%], and 11 [5.4%] in nonsepsis). Sepsis subjects had significantly more culture-proven infection and bacteremia than those who did not have sepsis.

Table 1.

Demographic and Clinical Characteristics of 409 Patients Who Had Blood Cultures Performed Classified by Sepsis Category

	Total	Nonsepsis	Sepsis
Variables	(n = 409)	(n = 203)	(n = 206)	P Value
Mean (SD) age, years	65.74 (17.84)	63.81 (17.66)	67.64 (17.85)	.03
Male gender, n (%)	210 (51.3)	97 (47.8)	113 (54.9)	.153
Comorbidities, n (%)	396 (96.8)	195 (96.1)	201 (97.6)	.383
Hypertension	230 (56.2)	110 (54.2)	120 (58.3)	.407
Diabetes mellitus	142 (34.7)	74 (36.5)	68 (33.0)	.465
Chronic kidney disease	99 (24.2)	52 (25.6)	47 (22.8)	.509
Dyslipidemia	82 (20.0)	42 (20.7)	40 (19.4)	.748
Nonhematologic malignancy	74 (18.1)	40 (19.7)	34 (16.5)	.401
Received immunosuppressive agent	67 (16.4)	25 (12.3)	42 (20.4)	.027
Heart disease	67 (16.4)	33 (16.3)	34 (16.5)	.946
Cerebrovascular disease	66 (16.1)	36 (17.7)	30 (14.6)	.383
Hematologic malignancy	49 (12.0)	19 (9.4)	30 (14.6)	.105
Heart failure	48 (11.7)	25 (12.3)	23 (11.2)	.718
Chronic lung disease	26 (6.4)	10 (4.9)	16 (7.8)	.239
Chronic liver disease	25 (6.1)	13 (6.4)	12 (5.8)	.807
Autoimmune disease	25 (6.1)	13 (6.4)	12 (5.8)	.807
Degenerative brain diseases	24 (5.9)	12 (5.9)	12 (5.8)	.970
HIV infection	9 (2.2)	4 (2.0)	5 (2.4)	.753
Transplant	8 (2.0)	6 (3.0)	2 (1.0)	.147
Others	128 (31.3)	65 (32.0)	63 (30.6)	.754
Culture proven, n (%)	237 (57.9)	89 (43.8)	148 (71.8)	<.001
Bacteremia, n (%)	117 (28.6)	29 (14.3%)	88 (42.7)	<.001
Type of Infection, n (%)
Hospital-acquired	242 (59.2)	116 (57.1)	126 (61.2)	.408
Community-acquired	141 (34.5)	73 (36.0)	68 (33.0)	.530
Healthcare-associated	26 (6.4)	14 (6.9)	12 (5.8)	.657
Site of Infection, n (%)
Respiratory tract	138 (33.7)	65 (32.0)	73 (35.4)	.465
Urinary tract	65 (15.9)	35 (17.7)	29 (14.1)	.312
Gastrointestinal tract	45 (11.0)	31 (15.0)	14 (6.9)	.008
Primary bacteremia	42 (10.3)	33 (16.0)	9 (4.4)	<.001
Skin and soft tissue	35 (8.6)	17 (8.3)	18 (8.9)	.824
Catheter-related BSI	12 (2.9)	6 (2.9)	6 (3.0)	.979
Cardiovascular	6 (1.5)	2 (1.0)	4 (2.0)	.401
Central nervous system	5 (1.2)	1 (0.5)	4 (2.0)	.172
Others	23 (5.6)	3 (1.5)	20 (9.9)	<.001
Unknown	60 (14.7)	21 (10.2)	39 (19.2)	.010
Severity of Illness
SIRS ≥2	323 (79.0)	131 (64.5)	192 (93.2)	<.001
qSOFA ≥2	155 (37.9)	38 (18.7)	117 (56.8)	<.001
SOFA ≥2	284 (69.4)	113 (55.7)	171 (83.0)	<.001
MEWS ≥4	288 (70.4)	105 (51.7)	183 (88.8)	<.001
NEWS ≥5	214 (52.3)	60 (29.6)	154 (74.8)	<.001
Receiving empiric ABT	386 (94.4)	184 (89.3)	202 (98)	.015
Concordant empiric ABT	146 (37.8)	54 (29.3)	92 (45.5)	.001
Combination empiric ABT	131 (33.9)	45 (24.5)	86 (42.5)	<.001
Receiving definite ABT	234 (57.2)	97 (47.8)	137 (66.5)	<.001
Concordant definite ABT	212 (91)	91 (94.8)	121 (88.3)	.183
Combination definite ABT	84 (35.9)	24 (24.7)	60 (43.8)	.003
Duration of ABT (days), median (IQR)	10 (6–15)	10 (7–15)	10 (6–15)	.434
Discharge Status, n (%)
Death	117 (28.6)	27 (13.3)	90 (43.7)	<.001

Abbreviations: ABT, antibiotic therapy; BSI, blood stream infection; HIV, human immunodeficiency virus; IQR, interquartile range; MEWS, modified early warning score; NEWS, national early warning score; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SD, standard deviation; SOFA, sequential organ failure assessment.

Performance of Various Scoring Systems for Sepsis Detection and Mortality Prediction

The performance of various scoring systems for sepsis detection is shown in Table 2. The SIRS ≥2 had the highest sensitivity (93.2%) but the lowest specificity (35.5%) for sepsis detection. In contrast, qSOFA ≥2 had a lower sensitivity (56.8%), but it had the highest specificity (81.3%) compared with SIRS ≥2, SOFA ≥2, MEWS ≥4, and NEWS ≥5. The qSOFA had the lowest false positive (FP) rate for detection of sepsis (18.7%) compared with the other scores. The FP rate of NEWS, MEWS, SOFA, and SIRS was 26.6%, 51.7%, 55.7%, and 64.5%, respectively. All scoring systems with recommended cutoff scores, including SIRS≥2, qSOFA ≥2, SOFA ≥2, MEWS ≥4, and NEWS ≥5, had moderate accuracy for sepsis detection with scores that ranged from 63.8% by SIRS to 72.6% by NEWS. The discriminatory power of all evaluated scores for detecting sepsis was analyzed by area under the ROC (AUROC) curve. That analysis revealed the highest AUROC for MEWS (AUROC, 0.79; 95% CI, 0.74–0.83), followed by NEWS (AUROC, 0.78; 95% CI, 0.74–0.83), SIRS (AUROC, 0.75; 95% CI, 0.70–0.79), qSOFA (AUROC, 0.72; 95% CI, 0.67–0.77), and SOFA (AUROC, 0.71; 95% CI, 0.66–0.76).

Table 2.

Diagnostic Performance Compared Among Various Sepsis Detection Scoring Systems

	Sensitivity	Specificity	PPV	NPV	Accuracy
Scoring System	% (95% CI)	% (95% CI)	% (95% CI)	% (95% CI)	% (95% CI)
SIRS ≥2	93.2 (88.9–96.2)	35.5 (28.9–42.5)	59.4 (53.9–64.8)	83.7 (74.2–90.8)	64.5 (59.7–69.2)
qSOFA ≥2	56.8 (49.7–63.7)	81.3 (75.2–86.4)	75.5 (67.9–82.0)	65.0 (58.7–70.8)	68.9 (64.2–73.4)
SOFA ≥2	83.0 (77.2–87.9)	44.3 (37.4–51.5)	60.2 (54.3–65.9)	72.0 (63.3–79.7)	63.8 (58.9–68.5)
MEWS ≥4	88.8 (83.7–92.8)	48.3 (41.2–55.4)	63.5 (57.7–69.1)	81.0 (72.9–87.6)	68.7 (64.0–73.2)
NEWS ≥5	74.8 (68.3–80.5)	70.4 (63.7–76.6)	72.0 (65.4–77.9)	73.3 (66.5–79.4)	72.6 (68.0–76.9)

Abbreviations: CI, confidence interval; MEWS, modified early warning score; NEWS, national early warning score; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.

The performance of the evaluated scoring systems for predicting mortality is shown in Table 3. All scoring systems had poor accuracy of lower than 60% for predicting mortality except for qSOFA score, which had an accuracy level of 70%. The discriminatory power of all evaluated scores for predicting hospital mortality was assessed by AUROC. The SOFA had the highest AUROC (AUROC, 0.76; 95% CI, 0.69–83), followed by qSOFA (AUROC, 0.74; 95% CI, 0.67–0.81), MEWS (AUROC, 0.72; 95% CI, 0.65–0.79), NEWS (AUROC, 0.70; 95% CI, 0.63–0.78), and SIRS (AUROC, 0.53; 95% CI, 0.45–0.61).

Table 3.

Performance Compared Among Various Scoring Systems for Predicting Mortality in Sepsis Patients

	Sensitivity	Specificity	PPV	NPV	Accuracy
Scoring System	% (95% CI)	% (95% CI)	% (95% CI)	% (95% CI)	% (95% CI)
SIRS ≥2	97.8 (92.2–99.7)	10.5 (5.3–18.0)	48.4 (40.9–55.9)	84.6 (54.6–98.1)	50.8 (43.5–58.0)
qSOFA ≥2	80.0 (70.2–87.7)	61.0 (50.9–70.3)	63.7 (54.1–72.6)	78.0 (67.5–86.4)	69.7 (62.8–76.1)
SOFA ≥2	94.4 (87.5–98.2)	25.7 (17.7–35.2)	52.1 (44.2–60.0)	84.4 (67.3–94.7)	57.4 (50.2–64.4)
MEWS ≥4	95.6 (89.0–98.8)	16.2 (9.7–24.7)	49.4 (41.8–57.1)	81.0 (58.1–94.6)	52.8 (45.6–59.0)
NEWS ≥5	86.7 (77.9–92.9)	33.3 (24.4–43.2)	52.7 (44.3–61.0)	74.5 (59.7–86.1)	57.9 (50.7–65.0)

Abbreviations: CI, confidence interval; MEWS, modified early warning score; NEWS, national early warning score; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.

The cutoff values of ≥3 for SIRS, ≥4 for SOFA, and ≥6 for MEWS could increase the accuracy (sensitivity, specificity) of sepsis detection to 70.2% (68.5%, 71.9%), 66% (62.1%, 69.9%), and 71.9% (63.1%, 80.8%), respectively. The cutoff values of ≥5 for SOFA, ≥8 for MEWS, and ≥7 for NEWS could increase the accuracy (sensitivity, specificity) of mortality prediction to 71.3% (66.7%, 75.2%), 68.7% (51.1%, 83.8%), and 67.2% (70%, 64.8%), respectively.

Causative Pathogens in Sepsis Subjects

Bacteria were responsible for infection in 89.3% of all sepsis subjects, consisting of Gram-negative bacteria in 64.8% and Gram-positive bacteria in 24.5%. Escherichia coli was the most commonly isolated Gram-negative bacteria (16.8%), followed by Acinetobacter baumannii (13.0%) and Pseudomonas aeruginosa (11.5%). Staphylococcus aureus was the most commonly isolated Gram-positive bacteria (8.6%), followed by Enterococcus faecium (5.3%) and Enterococcus faecalis (2.4%). Fungus was identified as a causative pathogen in 8.7% of subjects. Candida spp (4.8%) was the most commonly identified fungi. Virus (1.5%) and mycobacteria (0.5%) were also identified in this study cohort. The causative pathogen isolated from patients classified by sepsis category was shown in Supplementary Table S2.

Treatment in Sepsis and Nonsepsis Subjects

The median time from blood culture collection to the first dose of antibiotics was 45 minutes (interquartile range, 0–3 hours), which was not significantly different between survivors and nonsurvivors. The information on concordance of ABT is shown in Table 1. Fluid resuscitation was significantly more often given in nonsurvivors (45.6% vs 14.3%, P < .001). Other treatments, such as vasoactive agent, corticosteroid, mechanical ventilator, and renal replacement therapy, were all significantly more often given to or used in nonsurvivors than in survivors (45.6% vs 13.3%, P < .001; 42.2% vs 15.2%, P < .001; 58.9% vs 22.9%, P < .001; and, 20% vs 8.6%, P < .001, respectively).

Comparison of Sepsis Subjects Between Those Who Died and Survived

Of the 206 sepsis patients, 11 with unknown outcome were excluded because they left the hospital against medical advice or were transferred to other hospitals, and 90 patients died in the hospital for an in-hospital case-fatality rate of 43.7%. Of those who died, 51 patients (56.7%) died during sepsis treatment, and 73 patients (81.1%) died within 28 days for a 28-day case-fatality rate of 35.4%. The outcomes of the remaining 195 patients who survived or died are shown in Table 4.

Table 4.

Comparison Between Sepsis Patients Who Survived and Dieda

Variables	Total (n = 195)	Survivors (n = 105)	Died (n = 90)	P Value
Mean (SD) age, years	67.64 (17.80)	65.25 (18.83)	70.43 (16.18)	.042
Male gender, n (%)	104 (53.3)	52 (49.5)	52 (57.8)	.249
Comorbidities, n (%)	190 (97.4)	101 (96.2)	89 (98.9)	.235
Hypertension	114 (58.5)	65 (61.9)	49 (54.4)	.292
Diabetes mellitus	66 (33.8)	41 (39.0)	25 (27.8)	.097
Chronic kidney disease	44 (22.6)	25 (23.8)	19 (21.1)	.653
Received immunosuppressive agent	41 (21.0)	18 (17.1)	23 (25.6)	.151
Dyslipidemia	38 (19.5)	20 (19.0)	18 (20.0)	.867
Nonhematologic malignancy	33 (16.9)	16 (15.2)	17 (18.9)	.498
Heart disease	31 (15.9)	17 (16.2)	14 (15.6)	.904
Hematologic malignancy	29 (14.9)	14 (13.3)	15 (16.7)	.514
Cerebrovascular disease	28 (14.4)	16 (15.2)	12 (13.3)	.705
Heart failure	21 (10.8)	9 (8.6)	12 (13.3)	.285
Chronic lung disease	16 (8.2)	9 (8.6)	7 (7.8)	.840
Autoimmune disease	12 (6.2)	6 (5.7)	6 (6.7)	.783
Chronic liver disease	12 (6.2)	5 (4.8)	7 (7.8)	.382
Neurological diseases	12 (6.2)	8 (7.6)	4 (4.4)	.358
HIV infection	4 (2.1)	3 (2.9)	1 (1.1)	.391
Transplant	2 (1.0)	1 (1.0)	1 (1.1)	.913
Others	59 (30.3)	34 (32.4)	25 (27.8)	.485
Culture proven, n (%)	140 (71.8)	71 (67.6)	69 (76.7)	.162
Bacteremia, n (%)	83 (42.6)	43 (41.0)	40 (44.4)	.755
Type of Infection, n (%)
Hospital-acquired	121 (62.1)	56 (53.3)	65 (72.2)	.007
Community-acquired	62 (31.8)	41 (39.0)	21 (23.2)	.019
Healthcare-associated	12 (6.2)	8 (7.6)	4 (4.4)	.358
Site of Infection, n (%)
Respiratory tract	70 (35.9)	27 (25.7)	43 (47.8)	.001
Primary bacteremia	33 (16.9)	15 (14.3)	18 (20.0)	.289
Gastrointestinal tract	30 (15.4)	17 (16.2)	13 (14.4)	.736
Urinary tract	27 (13.8)	20 (19)	7 (7.8)	.023
Skin and soft tissue	15 (7.7)	10 (9.5)	5 (5.6)	.300
Catheter-related BSI	4 (2.1)	2 (1.9)	2 (2.2)	.876
Cardiovascular	2 (1.0)	1 (1.0)	1 (1.1)	.913
Central nervous system	1 (0.5)	1 (1.0)	0 (0.0)	.353
Others	3 (1.5)	2 (1.9)	1 (1.1)	.654
Unknown	19 (9.7)	13 (12.4)	6 (6.7)	.180
Number of Pathogens, n (%)
Single pathogen	98 (68.1)	52 (72.2)	46 (63.9)	.284
Mixed pathogen	46 (31.9)	20 (27.8)	26 (36.1)	.284
Severity of Illness
Septic shock, n (%)	45 (23.1)	10 (9.5)	35 (38.9)	<.001
SIRS ≥2	182 (93.3)	94 (89.5)	88 (97.8)	.021
qSOFA ≥2	113 (57.9)	41 (39.0)	72 (80.0)	<.001
SOFA ≥2	163 (83.6)	78 (74.3)	85 (94.4)	<.001
MEWS ≥4	174 (89.2)	88 (83.8)	86 (95.6)	.008
NEWS ≥5	148 (75.9)	70 (66.7)	78 (86.7)	.001
Receiving empiric ABT	191 (97.9)	103 (98.1)	90 (100)	.128
Concordant empiric ABT	89 (46.6)	50 (48.5)	39 (44.3)	.560
Receiving definite ABT	133 (68.2)	74 (70.5)	59 (65.6)	.606
Concordant definite ABT	118 (88.7)	66 (89.2)	52 (88.1)	.530
Duration of ABT (days)	10 (6–15)	12 (7–16)	8 (3–15)	<.001
Median (IQR)
Length of stay (days), median (IQR)	20 (6.5–39)	21 (8–42)	17 (5–36)	.394

Abbreviations: ABT, antibiotic therapy; BSI, blood stream infection; HIV, human immunodeficiency virus; IQR, interquartile range; MEWS, modified early warning score; NEWS, national early warning score; qSOFA, quick sepsis-related organ failure assessment; SD, standard deviation; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment.

aExcluded 11 patients who left the hospital due to transfer or against medical advice.

The results of multivariate analysis for factors independently associated with death among sepsis patients are shown in Table 5. The independent factors associated with in-hospital mortality included presence of septic shock, qSOFA ≥2, and SOFA ≥2.

Table 5.

Multivariate Analysis for Factors Associated With Mortality in Sepsis Patients

Factors	Crude OR (95% CI)	P Value	Adjusted ORa (95% CI)	P Value
Age	1.02 (1.001–1.03)	.044	-	-
Hospital-acquired infection	2.28 (1.25–4.14)	.007	-	-
Respiratory tract infection	2.64 (1.45–4.83)	.002	-	-
Urinary tract infection	0.36 (0.14–0.89)	.028	-	-
Septic shock	6.45 (2.78–13.15)	<.001	4.72 (1.98–11.25)	<.001
SIRS ≥2	5.15 (1.11–23.88)	.036	7.86 (1.33–46.59)	.023
qSOFA ≥2	6.24 (3.27–11.94)	<.001	3.75 (1.84–7.65)	<.001
SOFA ≥2	5.89 (2.16–16.04)	.001	3.68 (1.19–11.41)	.024
MEWS ≥4	4.15 (1.34–12.85)	.013	-	-
NEWS ≥5	3.25 (1.57–6.75)	.002	-	-
Total duration of ABT	0.95 (0.91–0.96)	.006	-	-

Abbreviations: ABT, antibiotic therapy; CI, confidence interval; OR, odds ratio; qSOFA, quick sepsis-related organ failure assessment; SIRS, systemic inflammatory response syndrome; SOFA, sequential organ failure assessment score.

aAdjusted for age, hospital-acquired infection, respiratory tract infection, urinary tract infection, septic shock, SIRS ≥2, qSOFA ≥2, SOFA ≥2, MEWS ≥, NEWS ≥5, and total duration of ABT.

DISCUSSION

To the best of our knowledge, this is the largest prospective cohort of sepsis patients in Thailand to be studied to determine the diagnostic and prognostic utility of various sepsis scoring systems. The early identification of sepsis is essential because earlier intervention improves patient outcomes. In the present study, we identified eligible subjects who might have sepsis from those who had blood culture performed because blood culture is recommended for all patients suspected of having sepsis and septic shock, bacteremia, or blood stream infection [14]. The presence of sepsis and septic shock was then determined by agreement between 2 ID specialists as the reference standard to evaluate the accuracy of the various aforementioned scoring systems. An approach to identifying sepsis in a patient who had blood culture performed is partly similar to that developed by the Centers for Disease Control and Prevention in the United States, called Adult Sepsis Event (ASE) [15]. The ASE is signified by obtaining blood culture and at least 4 consecutive days of antibiotics starting within 2 calendars days of when blood culture was obtained plus evidence of concurrent organ dysfunction.

In the present study, among the patients who had blood cultures performed, 43% of them were diagnosed as sepsis and one third had bacteremia. Approximately 36% and 44% of sepsis patients died within 28 days and in the hospital, respectively. Almost 60% of all death occurred during sepsis treatment. Regarding sepsis detection, SIRS ≥2 criteria had the highest sensitivity (93.2%) for sepsis detection; therefore, this might be a good screening tool for sepsis detection. Although qSOFA ≥2 had a lower sensitivity of 56.8%, it had the highest specificity (81.3%) compared with the other scoring systems. Therefore, qSOFA ≥2 was more accurate in sepsis detection than the other scores, with the exception of NEWS ≥5. When considering the AUROC, which calculated all possible cut-point values between sepsis and nonsepsis for each scoring system, all scores included in our study demonstrated good discrimination between sepsis and nonsepsis (AUC ≥0.7), especially MEWS ≥4 and NEWS ≥5. Although our results revealed that NEWS ≥5 exhibited the highest accuracy and almost the highest discriminatory ability for detecting sepsis, the relatively complicated calculation of this score could limit its use in daily clinical practice. Our study finding is consistent with that from a recent meta-analysis that found SIRS to be more sensitive but less specific than qSOFA for diagnosis of sepsis [16].

Concerning mortality prediction, SIRS ≥2 criteria had the highest sensitivity (97.8%) but unacceptably low specificity (10.5%). It is interesting to note that the other scores also had high sensitivity but lower specificity. Among those, qSOFA ≥2 had a high sensitivity (80%) with fair specificity (61%), which resulted in the highest accuracy for predicting mortality (69.7%). SOFA ≥2 was less accurate than qSOFA ≥2 for predicting mortality. However, an increase in the cut-point for SOFA from 2 or greater to 5 or greater could increase the accuracy to 71%. When considering the AUROC, which calculated all possible cut-point values between sepsis and nonsepsis for each scoring system, all assessments except for SIRS had good discriminatory power for predicting mortality (AUC ≥0.7). The SOFA and qSOFA outperformed the other scores.

Our results are similar to those from recent retrospective studies that included 2350 patients. Those studies found SOFA to be a better prognostic tool for predicting mortality and organ failure than qSOFA and SIRS among sepsis patients admitted to the ICU [17, 18]. However, a study in patients with suspected infection outside the ICU found qSOFA ≥2 to be a better predictor of mortality than SOFA ≥2 and SIRS ≥2 [8]. Other studies in other settings, including the emergency department [9], and a multicenter study from LMICs [19] also found qSOFA to be the best scoring system for predicting mortality. Similar findings were also observed in 2 recent systematic reviews and meta-analyses [16, 20].

Scoring systems each have their own strengths and limitations. For example, using SIRS, which had the highest sensitivity, might be most appropriate system for urgent screening of sepsis, especially in the emergency department. However, to fulfill the score, complete blood count is needed, which is time-consuming. In contrast, the poor sensitivity but high specificity of qSOFA for sepsis detection without any laboratory parameters may facilitate the delivery of early and appropriate interventions, especially among patients with a qSOFA ≥2. The MEWS and NEWS criteria have only clinical parameters, but they are more complex and less user-friendly scoring systems compared with qSOFA score. These obstacles limit their role in routine clinical practice, especially among physicians who work in urgent and high-throughput settings.

The strength of this study is that we evaluated adult sepsis patients from almost all units in the hospital; however, pediatric patients were excluded. Therefore, the results of this study should be considered generalizable to different adult patient populations in resource-limited settings. This study also has some mentionable limitations. First, because we used patients who had blood culture performed as a surrogate for suspected sepsis as our initial screening for the enrollment of subjects, our study could have in some way been influenced by selection bias. Some cases with sepsis or septic shock might have died early, or those who decided to receive palliative care without any treatment might not have had blood taken for culture. These could have led to an underestimation of hospitalized patients with sepsis. Second, we evaluated the score at the time of or within 6 hours before blood draw for culture, which might be later than the actual time of sepsis onset. Moreover, the sepsis score could have changed over time during the evolution of sepsis. Finally, there is currently no universally acceptable gold standard for sepsis diagnosis. Our study used the collective opinion of 2 ID specialists who reviewed all information of each subject from the time of presentation to the hospital until the patient died or was discharged from the hospital for sepsis diagnosis as the reference standard. This remains subjective and imperfect; however, there was substantial concordance between the opinions of the 2 ID specialists. In addition, some factors might have influenced the opinion of the ID specialists, such as overdiagnosis bias in cases with known blood culture results.

CONCLUSIONS

The SIRS ≥2 was the most sensitive, qSOFA ≥2 was the most specific, and NEWS ≥5 was the most accurate sepsis detection scoring system. The qSOFA ≥2 was the most accurate system for predicting hospital mortality in sepsis patients. The appropriate application of each scoring system is useful for different specific purposes relative to early detection and mortality prediction in sepsis.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.

Click here for additional data file.