Comparison of the quick Sepsis-related Organ Failure Assessment and adult sepsis pathway in predicting adverse outcomes among adult patients in general wards: a retrospective observational cohort study.

BACKGROUND: Quick Sepsis-related Organ Failure Assessment (qSOFA) is recommended for use by the most recent international sepsis definition taskforce to identify suspected sepsis in patients outside the intensive care unit (ICU) at risk of adverse outcomes. Evidence of its comparative effectiveness with existing sepsis recognition tools is important to guide decisions about its widespread implementation. AIM: To compare the performance of qSOFA with the adult sepsis pathway (ASP), a current sepsis recognition tool widely used in NSW hospitals and systemic inflammatory response syndrome criteria in predicting adverse outcomes in adult patients on general wards.
METHODS: A retrospective observational cohort study was conducted which included all adults with suspected infections admitted to a Sydney teaching hospital between December 2014 and June 2016. The primary outcome was in-hospital mortality with two secondary composite outcomes.
RESULTS: Among 2940 patients with suspected infection, 217 (7.38%) died in-hospital and 702 (23.88%) were subsequently admitted to ICU. The ASP showed the greatest ability to correctly discriminate in-hospital mortality and secondary outcomes. The area under the receiver-operating characteristic curve for mortality was 0.76 (95% confidence interval (CI): 0.74-0.78), compared to 0.64 for the qSOFA tool (95% CI: 0.61-0.67, P < 0.0001). Median time from the first ASP sepsis warning to death was 8.21 days (interquartile range (IQR): 2.29-16.75) while it was 0 days for qSOFA (IQR: 0-2.58).
CONCLUSIONS: The ASP demonstrated both greater prognostic accuracy and earlier warning for in-hospital mortality for adults on hospital wards compared to qSOFA. Hospitals already using ASP may not benefit from switching to the qSOFA tool.

Introduction

Sepsis, defined as ‘life‐threatening organ dysfunction caused by a dysregulated host response to infection’, is estimated to affect 20–30 million patients each year worldwide. Despite advances in vaccines, antibiotics and acute care, sepsis remains the primary cause of death from infection. Between 30 and 50% of patients with sepsis die. , Early recognition allows for prompt treatment, which has been shown to be associated with reduced mortality. Sepsis recognition tools can play an important role in facilitating early sepsis diagnosis and initiation of treatment.

The Third International Consensus Definition for Sepsis and Septic Shock (SEPSIS‐3) was published in 2016. , The task force for SEPSIS‐3 advocated the quick Sequential Organ Failure Assessment (qSOFA) score to be used as a tool to recognise adult patients with suspected sepsis at risk of adverse outcomes outside the intensive care unit (ICU). , These patients can be rapidly identified if they have an infection and at least two of the following three clinical criteria: respiratory rate of 22/min or greater, altered mentation, or systolic blood pressure of 100 mmHg or less, which places them at particular risk of adverse outcomes, including prolonged ICU stay and mortality. The use of the qSOFA is a significant departure from the older systemic inflammatory response syndrome (SIRS) criteria, introduced in the first two international sepsis consensus definitions. , Several sepsis recognition tools have been developed based on the SIRS criteria and are currently in use in many countries.

The adult sepsis pathway (ASP) , is one of the most widely implemented sepsis recognition tools in Australia. The ASP tool was developed based on criteria used within the NSW deteriorating patient programme (Between the Flags) for recognising and responding to a deteriorating patient. , It is currently used in more than 180 hospitals in New South Wales to assist recognition of suspected sepsis cases among adult patients, and to facilitate treatment and appropriate referral to specialist teams. Robust assessment of the performance of the newer qSOFA tool in predicting adverse outcomes of sepsis patients in comparison with existing sepsis recognition tools, such as the ASP, is important in determining its suitability for widespread adoption.

The performance of the sepsis recognition tools, including the newer qSOFA tool, may vary by clinical setting. While the research attention has focussed on the early identification and treatment of patients with sepsis in emergency departments (ED) and ICU, many patients admitted to hospital general care wards acquire sepsis or deteriorate after initial ED diagnosis and treatment. Previous studies indicate that only 32% to 50% of patients with sepsis require ICU care, and the majority of patients with sepsis are not transferred to an ICU but cared for on general wards. , , Limited evidence exists about the patient outcomes associated with using sepsis recognition tools on general wards. We conducted a retrospective analysis of a large cohort of general ward patients. Our aim was to compare the performance of qSOFA, with the ASP and SIRS recognition criteria, in predicting mortality and extended ICU length of stay. We further aimed to compare differences in the timing of sepsis warnings triggered by these three tools.

Methods

Study design, setting and population

We conducted a retrospective observational cohort study, which included all adult patients (≥18 years) admitted between December 2014 and June 2016 to a 570‐bed metropolitan tertiary referral hospital in Sydney. Patients were excluded if they: (i) had a principal diagnosis of pregnancy and/or childbirth; (ii) were admitted directly to ICU; or (iii) had no vital signs or laboratory data documented during their admission (Fig. 1). Ethics approval was provided by the Macquarie University Human Research Ethics Committee (reference no: 5201600265).

Figure 1

Flow diagram of study patients.

Defining suspected infection

A patient was defined as having a suspected infection if any blood culture was ordered during the patient's hospital stay. Three other definitions were applied in sensitivity analyses using different surrogate markers: (i) ordering of two or more blood cultures; (ii) any blood culture and C‐reactive protein/procalcitonin (CRP/PCT); and (iii) any blood culture or CRP/PCT.

Suspected sepsis identified using sepsis recognition tools

A suspected sepsis case was identified when a sepsis warning was triggered based on a recognition tool among patients with suspected infection. The criteria for triggering a sepsis warning for each of the three sepsis recognition tools, that is the ASP, qSOFA and SIRS, are listed in Table 1.

Table 1

Three sepsis recognition tools

Measurements	qSOFA criteria 6 (SOS warning: at least two criteria are met)	SIRS criteria 7 (SOS warning: at least two criteria are met)	Adult sepsis pathway 9
			Red Zone (SOS warning: at least one criterion is met)	Yellow Zone (SOS warning: at least two criteria are met)
Systolic blood pressure (mmHg)	≤100		<90	<100
Respiratory rate (breaths/min)	≥22	>20		≤10 or ≥25
Glasgow Coma Scale score	<15			<15
Temperature (°C)	—	>38 or <36		<35.5 or >38.5
Heart rate (b.p.m.)	—	>90		≤50 or ≥120
White blood cell count (/mm3)	—	>12 000 or <4000	—	—
Lactate (mmol/L)			≥4.0	≥2.0
Base excess (mEq/L)			<−5.0
SpO2 (%)				<95

—, not included as part of criteria; qSOFA, quick Sepsis‐related Organ Failure Assessment; SIRS, systemic inflammatory response syndrome; SOS, suspicion of sepsis; SpO2, peripheral oxygen saturation.

The ASP tool provides two levels of sepsis warnings: (i) Red Zone for patients with suspected severe sepsis/septic shock; and (ii) Yellow Zone for patients who have suspected sepsis (Table 1).

Data sources and linkage

This study utilised routinely collected hospital clinical and administrative datasets. Patient demographic, admission and clinical data, including age, gender, vital signs, laboratory results, blood cultures and ICU admissions, were extracted from various clinical information systems, including Patient Admission Systems and Laboratory Information Systems. More than 3.7 million time‐stamped vital signs and laboratory test results were included. Data sets from these sources were linked using de‐identified medical record numbers and time stamp data where relevant.

Outcomes

The primary outcome of the study was in‐hospital mortality, and two secondary outcomes were composite outcomes: (i) in‐hospital mortality or ICU admission; and (ii) in‐hospital mortality or prolonged ICU stay (≥3 days), which was previously defined in the study on assessing clinical criteria for SEPSIS‐3.

Statistical analysis

Algorithms which encapsulated the criteria for each sepsis recognition tool (Table 1) were developed. A sepsis score was calculated for each relevant vital sign or laboratory test result during hospital stays (excluding ED and/or ICU stays). A relevant sepsis warning was produced if the score met the minimum sepsis score for a recognition tool, for example a qSOFA score ≥2 would result in a qSOFA sepsis warning, and an ASP warning from either Red or Yellow Zone would lead to an overall ASP warning. Patients might experience multiple sepsis warnings during their admissions. If ASP warnings from both Red and Yellow Zones were triggered during a hospital stay, the ASP warning level was recorded as a Red Zone warning. No patients were excluded in any analysis due to lack of other laboratory parameters required for the SIRS or ASP tools.

The performance of each of the three tools in predicting study outcomes for ward patients were compared. Comparisons were made using sensitivity, specificity, positive predictive value, and negative predictive value. The area under the receiver‐operating characteristic curve (AUROC) was used to assess the discrimination of each tool, that is the ability of each tool to correctly predict those with and without the study outcomes. The non‐parametric approach of DeLong for comparing AUROC was applied to take into account the correlated nature of the data. Sensitivity analyses were conducted for the primary outcome based on: (i) three additional definitions for suspected infection; and (ii) regardless of the infection status, that is including all eligible patients.

A secondary analysis of two levels of the ASP warning was also conducted. Patient characteristics were compared between cohorts with two levels of the ASP warnings using t‐tests, Wilcoxon‐Mann–Whitney tests and χ2 tests as appropriate based on the distribution of the data.

The median time from first sepsis warning to the primary outcome, that is in‐hospital death, was presented for each recognition tool. We plotted the cumulative proportions of patients who died in hospital in one‐hour increments after the first warning of suspected sepsis by each tool. Analyses were performed using R (version 3.5.0; R Foundation for Statistical Computing, Vienna, Austria) and SAS (version 9.4; SAS Institute Inc., Cary, NC, USA).

Results

A total of 42 870 patient admissions was recorded during the study period, of which 28 978 ward admissions were eligible for inclusion (Fig. 1). The final study cohort consisted of 2940 patients who had a suspected infection during their stay (Table 2). A total of 702 (23.88%) of these patients had an ICU admission and 217 (7.38%) died in hospital (Table 2).

Table 2

Characteristics of study patients and suspected sepsis patients recognised by three sepsis recognition tools

Characteristic	All study patients (n = 2940)	qSOFA score ≥ 2 (n = 332, 11.29%†)	SIRS score ≥ 2 (n = 2054, 69.86%†)	Adult sepsis pathway
				Overall (n = 1305, 44.39%†)	Yellow zone (n = 669, 22.75%†)	Red Zone (n = 636, 21.63%†)	P‐value‡
Age, median (IQR) (years)	69 (53–80)	75 (60–83)	70 (55–81)	71 (59–81)	72 (60–82)	71 (57–81)	0.65
Male sex, n (%)	1442 (49.05)	174 (52.41)	985 (47.96)	660 (50.57)	339 (50.67)	321 (50.47)	0.94
ATSI, n (%)	96 (3.27)	8 (2.41)	61 (2.97)	41 (3.14)	24 (3.59)	17 (2.67)	0.34
ICU admissions, n (%)	702 (23.88)	99 (29.82)	475 (23.13)	535 (41.00)	162 (24.22)	373 (58.65)	<0.0001
ICU LOS, median (IQR) (h)	83 (34–169)	92 (39–198)	68 (26–142)	78 (35–165)	70 (27–137)	85 (38–188)	<0.0001
Hospital LOS, median (IQR) (days)	8.25 (4.66–15.79)	14.12 (7.73–24.28)	10.07 (5.55–18.28)	12.12 (6.68–21.03)	11.33 (6.22–18.81)	12.86 (6.95–24.09)	<0.0001
In‐hospital mortality, n (%) (died on wards + died in ICU)	217 (7.38) (103 + 114)	80 (24.10) (51 + 29)	163 (7.94) (98 + 65)	202 (15.48) (93 + 109)	47 (7.03) (36 + 11)	155 (24.37) (57 + 98)	<0.0001
Composite outcome 1§, n (%)	805 (27.38)	150 (45.18)	573 (27.90)	628 (48.12)	198 (29.60)	430 (67.61)	<0.0001
Composite outcome 2§, n (%)	519 (17.65)	125 (37.65)	383 (18.65)	463 (35.48)	129 (19.28)	334 (52.52)	<0.0001

Percentages were calculated out of 2940 patients with suspected infection.

P‐values were obtained to compare between patients recognised with Red Zone sepsis and those with Yellow Zone sepsis using the adult sepsis pathway.

Composite outcome 1 = In‐hospital mortality or ICU admission; composite outcome 2 = In‐hospital mortality or prolonged ICU stay (≥3 days).

ATSI, Aboriginal and/or Torres Strait Islander origin; ICU, intensive care unit; IQR, interquartile range; LOS, length of stay; qSOFA, quick Sepsis‐related Organ Failure Assessment; SIRS, systemic inflammatory response syndrome.

Patients identified with suspected sepsis by the sepsis recognition tools

Among these 2940 patients, 69.86% (n = 2054) would be recognised as suspected sepsis cases based on SIRS criteria, 44.39% (n = 1305) on ASP and 11.29% (n = 332) on qSOFA criteria (Fig. 1).

Patients with suspected sepsis based on the qSOFA tool were more likely to be older, male, have a longer ICU and overall hospital stay, and higher in‐hospital mortality rate compared to patients meeting suspected sepsis criteria for the SIRS or ASP (Table 2).

Of all 217 deaths that occurred in the study period, a greater proportion occurred in ICU (52.53%, n = 114) than on wards (47.47%, n = 103). Among suspected sepsis patients identified using ASP, nearly half died on wards (46% = 98/202) (Table 2).

Prediction of adverse outcomes for patients on general wards

The ASP tool performed best in terms of predicting in‐hospital mortality among ward patients with suspected infection based on whether an overall ASP warning was triggered, with AUROC 0.76 (95% confidence interval (CI): 0.74–0.78), followed by the qSOFA tool (0.64, 95% CI: 0.61–0.67), and SIRS (0.53, 95% CI: 0.50–0.56) (P < 0.0001 for all pairwise comparisons; Fig. 2). ASP also performed best for the two secondary composite outcomes (P < 0.0001 for all pairwise comparisons of three tools within each outcome measure).

Figure 2

Comparison of the discriminatory capacities of three sepsis recognition tools for in‐hospital mortality and two composite outcomes: Composite outcome 1 = in‐hospital mortality or intensive care unit (ICU) admission; composite outcome 2 = in‐hospital mortality or prolonged ICU stay (≥3 days) (n = 2940). Dots represent point estimates, and error bars represent 95% confidence intervals. AUROC, area under receiver operating characteristic curve.

The ASP had the highest sensitivity across the three outcome measures, followed by SIRS and qSOFA (Table 3). The qSOFA tool had the highest specificity across the three outcome measures, followed by the ASP and SIRS.

Table 3

Performance of the three sepsis recognition tools by patient outcomes (n = 2940)

Patient outcome	Sepsis recognition tool	Sensitivity, % (95% CI)	Specificity, % (95% CI)	PPV, % (95% CI)	NPV, % (95% CI)
In‐hospital mortality	qSOFA	36.87 (30.45–43.29)	90.75 (89.66–91.83)	24.10 (19.5–28.70)	94.75 (93.89–95.60)
	SIRS	75.12 (69.36–80.87)	30.55 (28.82–32.28)	7.94 (6.77–9.10)	93.91 (92.33–95.48)
	ASP – overall	93.09 (89.71–96.46)	59.49 (57.65–61.34)	15.48 (13.52–17.44)	99.08 (98.62–99.54)
	ASP – yellow	21.66 (16.18–27.14)	76.94 (75.36–78.52)	6.96 (5.04–8.88)	92.49 (91.41–93.58)
	ASP – red	71.43 (65.42–77.44)	82.34 (80.9–83.77)	24.37 (21.03–27.71)	97.31 (96.65–97.97)
Composite outcome 1†	qSOFA	18.63 (15.94–21.32)	91.48 (90.29–92.66)	45.18 (39.83–50.53)	74.89 (73.22–76.55)
	SIRS	71.18 (68.05–74.31)	30.63 (28.68–32.59)	27.90 (25.96–29.84)	73.81 (70.92–76.71)
	ASP – overall	78.01 (75.15–80.87)	68.29 (66.32–70.26)	48.12 (45.41–50.83)	89.17 (87.67–90.68)
	ASP – yellow	25.22 (22.22–28.22)	77.89 (76.13–79.65)	30.07 (26.61–33.53)	73.42 (71.6–75.24)
	ASP – red	53.42 (49.97–56.86)	90.35 (89.1–91.6)	67.61 (63.97–71.25)	83.72 (82.22–85.23)
Composite outcome 2†	qSOFA	24.08 (20.41–27.76)	91.45 (90.34–92.56)	37.65 (32.44–42.86)	84.89 (83.52–86.27)
	SIRS	73.8 (70.01–77.58)	30.98 (29.14–32.82)	18.65 (16.96–20.33)	84.65 (82.28–87.02)
	ASP – overall	89.21 (86.54–91.88)	65.22 (63.32–67.12)	35.48 (32.88–38.07)	96.57 (95.69–97.46)
	ASP – yellow	25.63 (21.87–29.38)	77.61 (75.95–79.27)	19.70 (16.70–22.70)	82.96 (81.41–84.51)
	ASP – red	64.35 (60.23–68.48)	87.53 (86.21–88.84)	52.52 (48.63–56.40)	91.97 (90.86–93.08)

Composite outcome 1 = In‐hospital mortality or intensive care unit (ICU) admission; composite outcome 2 = In‐hospital mortality or prolonged ICU stay (≥3 days).

ASP, adult sepsis pathway; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick Sepsis‐related Organ Failure Assessment; SIRS, systemic inflammatory response syndrome.

Time from first sepsis warning to in‐hospital death

Of the 217 patients who died in hospital, most of them (93.09%, n = 202) would have received an overall ASP sepsis warning during their hospital stays; 75.12% (n = 163) with a SIRS warning; and only 36.87% (n = 80) with a qSOFA warning. Overall, ASP warnings for these patients would have occurred earlier than the SIRS or qSOFA warnings. The median time from a patient first meeting the ASP criteria to death was 8.21 days (interquartile range (IQR): 2.29–16.75), followed by SIRS (6.42 days, IQR: 0.29–17.08) and qSOFA (0 days, IQR: 0–2.58). Over time, a greater proportion of patients who died would have been recognised by the ASP compared to the use of the other two tools (Fig. 3). For example, more than 25% of patients would have received an ASP sepsis warning 4 days (96 h) before death, compared to less than 16% from the other two tools.

Figure 3

Cumulative proportions of 217 patients who died in hospital would have received a sepsis warning using the three sepsis tools (time intervals longer than 120 h (5 days) were trimmed). (), Quick Sepsis‐related Organ Failure Assessment; (), systemic inflammatory response syndrome; (), adult sepsis pathway.

Comparison of two levels of ASP warning

Among 1305 suspected septic patients identified using ASP, 636 (48.74%) had a Red Zone warning and 669 (51.26%) had a Yellow Zone warning (Table 2). Patients who experienced these two warnings had similar age and gender composition, but those with a Red Zone warning were more likely to be admitted to ICU and/or die after a much longer stay in ICU and hospital. Overall, the Red Zone warning had much better predictive validity, that is higher AUROCs, across all three outcomes than the Yellow Zone sepsis warning (Fig. 2).

Discussion

We compared the performance of three sepsis recognition tools in predicting adverse outcomes. We applied an innovative data‐driven process utilising a large volume of clinical data routinely collected as part of various clinical information systems in a large Australian hospital. Our findings provide support for the use of the ASP among hospital general ward patients, over qSOFA. We found that the ASP outperformed the qSOFA and SIRS tools in predicting patients at risk of experiencing typical adverse outcomes from sepsis, but also presented a significantly earlier sepsis warning for patients who died than either SIRS or qSOFA. The qSOFA demonstrated low sensitivity, identifying only 36.87% of patients with suspected infection who later died in hospital. In contrast, the ASP was successful at recognising 93.09% of these patients. A sepsis recognition tool with low sensitivity could result in missed cases and potentially lead to increased mortality rates.

A sepsis recognition tool generating an earlier warning may facilitate early treatment, which has been shown to be associated with reduced mortality. Seymour et al. found, in a sample of 49 331 patients with suspected infection from 149 US ED, that every hour delay in the administration of antibiotics was associated with a 4% significant increase in risk‐adjusted mortality. We found that about half of patients who died would have had an ASP warning 8.21 days before death while qSOFA warning would not have been triggered for two‐thirds of patients before death. These findings are consistent with those from another study of ward patients by Churpek et al. published in 2017. That study found almost half of patients who had an adverse outcome (i.e. death or prolonged ICU stay) did not have a qSOFA warning prior to the outcome.

While attention has been placed on the identification of sepsis among ED and ICU patients, , , previous studies have showed that the majority of patients with sepsis are treated on hospital general care wards. , These studies also reveal mortality rates of 26% to 30% among patients with sepsis who are not admitted to an ICU compared to 11–33% for those in the ICU. , We found that nearly half of all patients with suspected sepsis identified using ASP died on general wards, which suggests that there could be a significant underappreciation of the potential risks of mortality from sepsis on general wards. This may reflect a problem with access to ICU beds as has been highlighted in previous studies. Two Australian studies found that ward patients with suspected sepsis identified using qSOFA or SIRS at the time of Rapid Response Reviews were associated with poor outcomes, that is in‐hospital mortality and longer length of stay. , Further investigation is warranted to improve patient outcomes for patients with suspected sepsis on hospital general wards.

A systematic review, published in 2018, compared the qSOFA and SIRS in predicting mortality and identified only three studies , , involving patients on general wards, all three published in 2017. Two of these studies reported the AUROC results for in‐hospital mortality comparing the qSOFA and SIRS tools. Although direct comparison is difficult due to the heterogeneity of study populations and methods used, our AUROC results for in‐hospital mortality (ASP 0.76, qSOFA 0.64 and SIRS 0.53) were slightly lower, but at a similar level to these two studies. Churpek et al. reported similar AUROC for qSOFA (0.69) and SIRS (0.67) while Finkelsztein et al. had higher AUROC for qSOFA (0.74) and lower result for SIRS (0.59).

The main strength of this study is the application of an innovative data‐driven approach to link and exploit routinely collected data from hospital clinical information systems. We developed algorithms to retrospectively evaluate the performance of three sepsis recognition tools using these rich data sets, including vital signs and laboratory test results. This data driven approach allowed us to test different tools without consuming the substantial resources which would be required to implement and trial these tools in the field. This is an efficient first step in assessing whether an intervention may provide clinical benefits prior to introduction. Further investigation, including prospective studies, after the implementation of a sepsis recognition tool would be beneficial to determine whether the tool significantly improves patient outcomes.

Clinical information systems provide a valuable mechanism for incorporating sepsis recognition algorithms which automatically generate alerts and provide decision support to guide appropriate, prompt treatment. However, adoption of such algorithms requires close attention to both the specificity and sensitivity of alerting to avoid the increasingly recognised problem of alert fatigue among clinical system users. Few evaluations of the effectiveness of such electronic decision support have been conducted. Many hospitals rely on paper‐based sepsis recognition tools, susceptible to transcription and interpretation errors, and highly reliant on vigilant and timely review by clinicians. In contrast, appropriately designed automated systems have the potential to decrease delays and increase the accuracy of sepsis detection.

Limitations

Our study had several limitations. First, this was a single‐centre study and therefore the results may not be generalisable to other settings. Second, there is no gold standard to determine when a patient is suspected to have infection. Blood culture ordering and antibiotic usage have been used as a flag for suspected infection in the study for developing the qSOFA criteria by Seymour et al. In our study, suspected infection was inferred if a blood culture was ordered because data on antibiotic usage were not available. In addition, blood culture ordering was endorsed by ASP, which was recommended to be used at the study hospital during the study period. To address this limitation, we conducted four sensitivity analyses using: (i) three cohorts based on additional definitions for suspected infection; and (ii) all 28 978 patients regardless of infection status (see the Methods section). Results from all four analyses (Tables A1, A2) were consistent with those in Table 3 comparing the three recognition tools. The ASP outperformed qSOFA in prognostic accuracy although in‐hospital mortality rates were different. Third, we did not have information on Not for Resuscitation (NFR) or limitation of care orders, which may have inflated our primary outcome. To be conservative, we conducted a subgroup analysis excluding all 103 patients who died on wards. The ASP tool still had the highest AUROC among the three tools (0.78, 95% CI: 0.75–0.80, Table A3), which was similar to that from the main analysis (0.76). Fourth, our study examined typical adverse outcomes from sepsis, which were used in the study for developing the qSOFA criteria. However, it is acknowledged that these adverse outcomes could also be caused by other critical clinical conditions.

Table A1

Characteristics of patient cohorts defined by four definitions of suspected infection

Characteristics	Patients with suspected infection were identified if during the hospital stay there was (were)
	Any BC order	>1 BC orders	Any BC and any CRP/PCT orders	Any BC or CRP/PCT order
Number of patient admissions	2940	1435	2767	10 827
Age in years – median (IQR)	69 (53–80)	69 (54–80)	69 (54–80)	68 (50–80)
Male sex – n (%)	1442 (49.05)	694 (48.36)	1358 (49.08)	5101 (47.11)
ATSI – n (%)	96 (3.27)	46 (3.21)	93 (3.36)	393 (3.63)
ICU admissions – n (%)	702 (23.88)	419 (29.20)	694 (25.08)	1164 (10.75)
ICU LOS in hours – median (IQR)	83 (34–169)	99 (34–219	84 (35–172)	62 (24–129)
Hospital LOS in days – median (IQR)	8.25 (4.66–15.79)	11.12 (6.05–20.39)	8.68 (4.90–16.09)	5.64 (3.15–10.25)
In‐hospital mortality – n (%, 95% CI)	217 (7.38, 6.44–8.33)	132 (9.20, 7.70–10.69)	210 (7.59, 6.6–8.58)	361 (3.33, 3.00–3.67)
Composite outcome 1– n (%, 95% CI)	805 (27.38, 25.77–28.99)	475 (33.10, 30.67–35.54)	793 (28.66, 26.97–30.34)	1382 (12.76, 12.14–13.39)
Composite outcome 2 – n (%, 95% CI)	519 (17.65, 16.27–19.03)	326 (22.72, 20.55–24.89)	512 (18.50, 17.06–19.95)	784 (7.24, 6.75–7.73)

Composite outcome 1 = in‐hospital mortality or ICU admission; composite outcome 2 = In‐hospital mortality or prolonged ICU stay (≥3 days).

ATSI, Aboriginal and/or Torres Strait Islander origin; CI, confidence interval; ICU, intensive care unit; IQR, interquartile range; LOS, length of stay.

Table A2

Performance of the three recognition tools on predicting in‐hospital mortality by different suspected infection definitions

Suspected infection identified	Recognition tools	Sensitivity, % (95% CI)	Specificity, % (95% CI)	PPV, % (95% CI)	NPV, % (95% CI)	AUROC (95% CI)
Any BC order	qSOFA	36.87 (30.45–43.29)	90.75 (89.66–91.83)	24.10 (19.50–28.70)	94.75 (93.89–95.6)	0.64 (0.61–0.67)
	SIRS	75.12 (69.36–80.87)	30.55 (28.82–32.28)	7.94 (6.77–9.10)	93.91 (92.33–95.48)	0.53 (0.5–0.56)
	ASP – overall	93.09 (89.71–96.46)	59.49 (57.65–61.34)	15.48 (13.52–17.44)	99.08 (98.62–99.54)	0.76 (0.74–0.78)
	ASP – yellow	21.66 (16.18–27.14)	76.94 (75.36–78.52)	6.96 (5.04–8.88)	92.49 (91.41–93.58)	0.51 (0.48–0.54)
	ASP – red	71.43 (65.42–77.44)	82.34 (80.90–83.77)	24.37 (21.03–27.71)	97.31 (96.65–97.97)	0.77 (0.74–0.8)
>1 BC orders	qSOFA	40.15 (31.79–48.51)	87.41 (85.61–89.21)	24.42 (18.71–30.14)	93.51 (92.13–94.9)	0.64 (0.59–0.68)
	SIRS	81.06 (74.38–87.74)	21.64 (19.41–23.88)	9.49 (7.78–11.20)	91.86 (88.80–94.92)	0.51 (0.48–0.55)
	ASP – overall	98.48 (96.4–100.57)	52.11 (49.4–54.82)	17.24 (14.55–19.94)	99.71 (99.3–100.11)	0.75 (0.74–0.77)
	ASP – yellow	23.48 (16.25–30.72)	73.29 (70.89–75.69)	8.18 (5.42–10.94)	90.44 (88.66–92.21)	0.52 (0.48–0.55)
	ASP – red	75 (67.61–82.39)	78.36 (76.12–80.59)	25.98 (21.58–30.39)	96.87 (95.82–97.92)	0.77 (0.73–0.81)
Any BC and any CRP/PCT orders	qSOFA	37.14 (30.61–43.68)	90.34 (89.2–91.49)	24.00 (19.36–28.64)	94.59 (93.7–95.49)	0.64 (0.60–0.67)
	SIRS	76.19 (70.43–81.95)	29.14 (27.37–30.9)	8.11 (6.91–9.32)	93.71 (92.02–95.40)	0.53 (0.50–0.56)
	ASP – overall	93.33 (89.96–96.71)	57.96 (56.05–59.87)	15.42 (13.44–17.41)	99.06 (98.58–99.55)	0.76 (0.74–0.78)
	ASP – yellow	21.90 (16.31–27.50)	76.22 (74.57–77.87)	7.03 (5.07–8.99)	92.24 (91.1–93.38)	0.51 (0.48–0.54)
	ASP – red	71.43 (65.32–77.54)	81.50 (80–83.01)	24.08 (20.72–27.43)	97.20 (96.5–97.9)	0.76 (0.73–0.8)
Any BC or CRP/PCT order	qSOFA	33.24 (28.38–38.10)	95.01 (94.6–95.43)	18.69 (15.68–21.71)	97.63 (97.34–97.93)	0.64 (0.62–0.67)
	SIRS	78.12 (73.85–82.38)	52.56 (51.6–53.52)	5.37 (4.76–5.98)	98.58 (98.27–98.89)	0.65 (0.63–0.68)
	ASP – overall	87.81 (84.44–91.19)	76.50 (75.69–77.32)	11.42 (10.24–12.6)	99.45 (99.29–99.61)	0.82 (0.80–0.84)
	ASP – yellow	27.15 (22.56–31.73)	85.30 (84.63–85.98)	5.99 (4.84–7.14)	97.14 (96.8–97.48)	0.56 (0.54–0.59)
	ASP – red	61.22 (56.19–66.25)	91.10 (90.55–91.64)	19.17 (16.90–21.44)	98.55 (98.31–98.79)	0.76 (0.74–0.79)
Regardless of infection status (all 28 978 eligible patients)	qSOFA	31.00 (26.77–35.24)	97.49 (97.31–97.67)	16.57 (14.08–19.06)	98.88 (98.75–99.00)	0.64 (0.62–0.66)
	SIRS	73.58 (69.54–77.62)	75.48 (74.98–75.98)	4.60 (4.12–5.08)	99.44 (99.34–99.54)	0.75 (0.73–0.77)
	ASP – overall	81.00 (77.41–84.60)	88.70 (88.34–89.07)	10.33 (9.33–11.32)	99.66 (99.59–99.73)	0.85 (0.83–0.87)
	ASP – yellow	31.00 (26.77–35.24)	97.49 (97.31–97.67)	16.57 (14.08–19.06)	98.88 (98.75–99.00)	0.60 (0.58–0.62)
	ASP – red	73.58 (69.54–77.62)	75.48 (74.98–75.98)	4.60 (4.12–5.08)	99.44 (99.34–99.54)	0.75 (0.73–0.77)

ASP, adult sepsis pathway; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick Sepsis‐related Organ Failure Assessment; SIRS, systemic inflammatory response syndrome.

Table A3

Performance of the three recognition tools on predicting in‐hospital mortality

Recognition tool	Sensitivity, % (95% CI)	Specificity, % (95% CI)	PPV, % (95% CI)	NPV, % (95% CI)	AUROC (95% CI)
qSOFA	25.44 (17.44–33.43)	90.75 (89.66–91.83)	10.32 (6.76–13.88)	96.67 (95.98–97.37)	0.58 (0.54–0.62)
SIRS	57.02 (47.93–66.11)	30.55 (28.82–32.28)	3.32 (2.53–4.12)	94.44 (92.92–95.95)	0.56 (0.52–0.61)
ASP – overall	95.61 (91.85–99.37)	59.49 (57.65–61.34)	8.99 (7.38–10.60)	99.69 (99.42–99.96)	0.78 (0.75–0.80)
ASP – yellow	9.65 (4.23–15.07)	76.94 (75.36–78.52)	1.72 (0.71–2.73)	95.31 (94.43–96.20)	0.57 (0.54–0.60)
ASP – red	85.96 (79.59–92.34)	82.34 (80.90–83.77)	16.93 (13.87–19.98)	99.29 (98.95–99.64)	0.84 (0.81–0.87)

ASP, adult sepsis pathway; AUROC, area under the receiver‐operating characteristic curve; CI, confidence interval; NPV, negative predictive value; PPV, positive predictive value; qSOFA, quick Sepsis‐related Organ Failure Assessment; SIRS, systemic inflammatory response syndrome.

Implications for clinicians, researchers and policy makers

The present study provides evidence that hospitals currently using the ASP may not gain any additional benefit in switching to the qSOFA tool. The ASP demonstrated greater prognostic accuracy and earlier warning for in‐hospital mortality for adult ward patients with suspected infection compared to the qSOFA and SIRS tools. Early detection of sepsis would increase the chances to facilitate the prompt involvement of senior clinicians to confirm the diagnosis and support rapid treatment with appropriate intravenous antibiotics and fluids. Improving the availability of tools that support clinicians to more efficiently incorporate sepsis risk information into their clinical workflows, such as electronic decision support, should be investigated.