Prognostic value of the Quick Sepsis-related Organ Failure Assessment (qSOFA) score among critically ill medical and surgical patients with suspected infection in a resource-limited setting.

Background: The Quick Sequential Organ Failure Assessment (qSOFA) score is a simple bedside tool validated outside of the intensive care unit (ICU) to identify patients with suspected infection who are at risk for poor outcomes.
Objectives: To assess qSOFA at the time of ICU referral as a mortality prognosticator in adult medical v. surgical patients with suspected infection admitted to an ICU in a resource-limited regional hospital in South Africa (SA).
Methods: We conducted a retrospective cohort study on adult medical or surgical patients that were admitted to an ICU in a resource-limited hospital in SA. We performed univariate and multivariable logistic regression and compared nested models using likelihood ratio test, and we calculated the area under the receiver operating characteristic curve (AUROC).
Results: We recruited a total of 1 162 (medical n=283 and surgical n=875) participants in the study who were admitted to the ICU with suspected infection. qSOFA at the time of ICU referral was highly associated with but poorly discriminant of in-ICU mortality among medical (odds ratio (OR) 2.60, 95% confidence interval (CI) 1.19 - 5.71; p=0.02; AUROC 0.60; 95% CI 0.53 - 0.67; p=0.02) and surgical (OR 2.74; 95% CI 1.73-4.36; p<0.001; AUROC 0.60; 95% CI 0.55 - 0.65; p=0.04) patients. qSOFA model performance was similar between medical and surgical subgroups (p≥0.26). Addition of qSOFA to a baseline risk factor model including age, sex, and HIV status improved the model discrimination in both subgroups (medical AUROC 0.64; 95% CI 0.56 - 0.71; p=0.049; surgical AUROC 0.69; 95% CI 0.64 - 0.74; p<0.0001).
Conclusion: qSOFA was highly associated with, but poorly discriminant for, poor outcomes among medical and surgical patients with suspected infection admitted to the ICU in a resource-limited setting. These findings suggest that qSOFA may be useful as a tool to identify patients at increased risk of mortality in these populations and in this context.

Background

Sepsis is a major global health burden with high morbidity and mortality.[[1]] Approximately 30% of all intensive care unit (ICU) patients have or develop sepsis,[[1]] defined as life-threatening organ dysfunction caused by a dysregulated host response to infection.[[2]] More than a third of these patients do not survive their stay in hospital.[[1]] The limited availability of ICU beds is a major problem in resource-limited settings.[[3]] The ability to identify patients most likely to benefit from ICU admission may help ensure the optimal use of these facilities.

The quick Sequential Organ Failure Assessment (qSOFA) score is a bedside assessment tool introduced in the Sepsis-3 guidelines[[2]] as a prognosticator of poor outcomes in patients with suspected infection, and validated outside but not inside of the ICU setting.[[4]] The qSOFA includes three clinical parameters: (i) blood pressure (threshold systolic blood pressure ≤100 mmHg); (ii) respiratory rate (threshold ≥22 breaths per minute); and (iii) mentation (threshold Glasgow Coma Score (GCS) ≤14). Initial evaluation indicated that the presence of two or more qSOFA points is associated with a greater risk of death or prolonged ICU stay in developed countries.[[2]]

The qSOFA score is increasingly being used and studied, following its inclusion in the Sepsis-3 guidelines of 2016. Studies have shown mixed results in different patient populations and contexts. It appears that the qSOFA score may have low sensitivity but high specificity as a prognosticator of mortality.[[5-12]] Age, sex and HIV status have been shown to independently predict mortality in patients with infection in the US and have been used in prior qSOFA validation studies in resource-limited settings.[[13]]

There is limited research on the prognostic value of qSOFA in resource-limited settings – initial results from developing countries[[13]] and African countries are encouraging.[[14-17]] A recent study in ICU patients with and without suspected infection showed that high qSOFA scores on admission were associated with an increased likelihood of in-ICU mortality in South Africa (SA).[[18]] Further studies are required to assess the role of qSOFA score as a predictor of mortality in these settings.

The prognostic role of qSOFA in medical v. surgical cohorts is yet to be determined since studies to date have been conducted on mixed cohorts of patients. Therefore, studies are needed to assess qSOFA prognostic efficacy in these differing populations.

Methods

Design

We performed a single-centre, retrospective cohort study to evaluate the prognostic validity of the qSOFA score at the time of ICU referral in adult medical v. surgical ICU patients admitted with suspected infection. We utilised the Integrated Critical Care Electronic Database (ICED)[[19]] at Edendale Hospital in Kwazulu-Natal Province, SA. Edendale Hospital is a regional-level public hospital with 900 beds that operates in a resource-constrained context. The ICU admits both surgical and medical patients. The ICU has capacity for nine patients but provides mechanical ventilatory support for a maximum of six patients due to staffing and equipment constraints. This study used an ICU outcomes analytic dataset assembled from the ICED database and used in prior published studies.[[3,18]]

ICED includes all referrals for ICU care at Edendale Hospital since 2014. Referrals come from the emergency department and wards, as well as from other district and community hospitals in the KwaZulu-Natal Department of Health geographic catchment area. Data entry into ICED is integrated into the normal daily workflow and clinical documentation of the ICU, ensuring complete population capture.

Data entry categories include demographics, admitting diagnosis and subspecialty classification, referral information, clinical data, triage decision, and initial management plan including antibiotic prescription. Organ support, mortality, length of stay, and discharge destination are recorded at the end of the ICU stay. All entries are date and time stamped. Data are not collected after ICU discharge.

Participants

All adult patients (≥18 years) admitted to the ICU from January 2014 to December 2018 with suspected infection were eligible for inclusion. ‘Suspected infection’ was defined as commencement or continuation of antibiotic therapy at the time of ICU admission. Patients accepted for admission but never physically transferred/ admitted to the ICU for any reason (i.e. due to bed unavailability or a change in clinical status) were excluded. Patients were identified as medical or surgical patients based on the referring specialty or as assigned by the ICU team, as recorded in the database.

Exposure and outcomes

The primary exposure was the qSOFA score at the time of ICU referral (i.e. before the commencement of ICU care). For primary analyses, qSOFA was treated as an ordinal variable with 0 - 1 (reference), 2, and 3 points, due to the small number of study subjects with qSOFA score of 0 in this critical care population, and prior research showed a stepwise association with mortality.[[18]] In sensitivity analyses, we treated qSOFA as dichotomous (<2 v. ≥2 points) as per the Sepsis-3 guidelines.[[2]] The primary outcome was in-ICU mortality, defined as death in the ICU or a palliative ICU discharge.

Statistical analysis

The baseline characteristics of the included patients were reported as mean (standard deviation (SD)) for continuous normally distributed variables; median and interquartile range (IQR) for data that were not normally distributed; and count (percent) for categorical and ordinal variables.

The proportion of patients with suspected infection who died in ICU in each category was calculated. To measure the association between qSOFA score and in-ICU mortality, we performed univariate and multivariable logistic regression analysis. We performed further analysis using a baseline risk model that included age, sex, and HIV status, which have been shown to be independently associated with mortality, to assess the additive prognostic benefit of qSOFA and allow comparison with related prior studies.[[13]] We compared nested models (i.e. baseline v. baseline plus qSOFA) using likelihood ratio tests.

Since defining ‘suspected infection’ based on antibiotic therapy alone risks including some patients without acute infection (e.g. those receiving antibiotics for post-surgical prophylaxis, chronic infections, infection suppression, or anti-inflammatory indications) in a sensitivity analysis, we repeated our primary analysis including only patients for whom infection was the primary ICU admission diagnosis. This improved specificity at the expense of sensitivity (e.g. missing patients with a non-infectious indication for ICU admission listed primarily but also with acute infection).

Discrimination was assessed by calculating the area under the receiver operating characteristic curve (AUROC). Differences between AUROCs for medical and surgical patient subgroups were determined using the χ² test. All analyses were stratified by medical v. surgical patients.

For all analyses, a p<0.05 was considered to be statistically significant, and the precision of estimates was reported with 95% confidence interval (CI).

Ethics and consent

Ethics (ref. no. BREC/00001595/2020) and ICU database (ref. no. BCA 211/14) approvals were granted by the University of KwaZulu-Natal Biomedical Research Ethics Committee (BREC). The study protocol was also approved by the Institutional Review Board of the University of Pennsylvania (Philadelphia, USA).

Results

A total of 2 119 adults were admitted to the ICU and screened for eligibility during the study period. A total of 1 162 participants (surgical n=875 and medical n=287) with infection were included in the analysis (Fig. 1). Patient characteristics are described in Table 1.

Fig. 1

Consort diagram. (*No suspected infection, n=929; unknown referring speciality, n=28.) († Missing data, n=13.)

Table 1

Patient characteristics

	Medical (n=287), n (%)*	Surgical (n=875), n (%)*
Age (years), mean (SD)	41.51 (15.59)	40.39 (15.97)
Female	145 (50.5)	410 (46.9)
Pre-ICU hospital LoS (days), median (IQR)	1 (0 - 1)	1 (0 - 3)
Mechanically ventilated†	161 (56.1)	533 (60.9)
HIV status
Negative	182 (63.4)	669 (76.5)
Positive, taking ART	73 (25.4)	165 (18.9)
Positive, not taking ART	32 (11.2)	41 (4.7)
Maximal respiratory rate, mean (SD)	31 (15)	27 (14)
Minimum systolic blood pressure (mmHg), mean (SD)	100 (20)	102 (20)
Glasgow coma score, mean (SD)	9 (5)	8 (5)
qSOFA scores
0 - 1	79 (27.5)	325 (37.1)
2	127 (44.2)	368 (42.1)
3	77 (26.8)	176 (20.1)
Missing	4 (1.4)	6 (0.7)
Deaths	72 (25.1)	148 (16.9)

ART = antiretroviral therapy

ICU = intensive care unit

IQR = interquartile range

LoS = length of stay

qSOFA = Quick Sepsis-related Organ Failure Assessment

SD = standard deviation

* Unless otherwise specified

† At time of ICU admission

The number of accepted patient referrals from the surgical disciplines were as follows: general surgery (54.1%; n=473), trauma (25.5%; n=223), orthopaedics (6.9%; n=60), obstetrics (6.4%; n=56), gynaecology (4.0%; n=35), burns (1.6%; n=14), anaesthesia (0.6%; n=5), urology (0.6%; n=5), otolaryngology (0.3%; n=3), and maxillofacial (0.1%; n=1).

Univariate and adjusted multivariate models demonstrating the associations between qSOFA and age, sex and HIV status with in-ICU mortality are shown in Table 2. In both the unadjusted and adjusted models for medical (unadjusted odds ratio (OR) 2.85; 95% confidence interval (CI) 1.31 - 6.18; p=0.01; adjusted OR 2.60; 95% CI 1.19 - 5.71; p=0.02) and surgical (unadjusted OR 2.88; 95% CI 1.83 - 4.54; p<0.001; adjusted OR 2.74; 95% CI 1.73 - 4.36; p<0.001) subgroups, a qSOFA score of 3 but not a qSOFA score of 2, was associated with increased odds of in-ICU mortality compared with a qSOFA score of 0 - 1.

Table 2

Association of qSOFA and baseline characteristics with in-ICU mortality among medical and surgical patients with infection

	Medical, OR (95% CI)	p-value	Surgical, OR (95% CI)	p-value
Univariate
qSOFA (ref: 0 - 1)
qSOFA = 2	1.88 (0.90 - 3.92)	0.09	0.98 (0.63 - 1.52)	0.93
qSOFA = 3	2.85 (1.31 - 6.18)	0.01	2.88 (1.83 - 4.54)	<0.001
Age (per 10 years)	0.92 (0.77 - 1.10)	0.34	1.23 (1.16 - 1.43)	<0.001
Male	0.73 (0.42 - 1.26)	0.26	0.55 (0.39 - 0.80)	0.001
HIV-positive	1.78 (1.03 - 3.09)	0.04	1.10 (0.72 - 1.66)	0.67
Adjusted
qSOFA (ref: 0 - 1)
qSOFA = 2	1.81 (0.86 - 3.79)	0.12	0.93 (0.59 - 1.45)	0.74
qSOFA = 3	2.60 (1.19 - 5.71)	0.02	2.74 (1.73 - 4.36)	<0.001
Age (per 10 years)	1.00 (0.98 - 1.01)	0.63	1.02 (1.01 - 1.04)	<0.001
Male	0.80 (0.46 - 1.39)	0.43	0.62 (0.43 - 0.92)	0.02
HIV-positive	1.56 (0.87 - 2.78)	0.14	1.01 (0.65 - 1.57)	0.97

OR = odds ratio

CI = confidence interval

qSOFA = quick Sequential Organ Failure Assessment

* Baseline model includes age, sex and HIV status

The AUROCs for the baseline model, qSOFA alone, and qSOFA with the baseline model were all poor (95% CI 0.59 - 0.69) and there were no differences between performance in the medical and surgical subgroups (all p≥0.26), but addition of qSOFA to the baseline model improved performance in both subgroups (both p<0.05) (Table 3, Fig. 2 and appendix eFigs 1 and 2). Sensitivity analyses with qSOFA treated as a dichotomous score (<2 v. ≥2) showed similar but attenuated relationships and performance (appendix eTables 1 and 2 (https://www.samedical.org/ file/1677) and appendix eFigs 3 and 4 (https://www.samedical.org/file/1678). When restricting to patients for whom infection was the primary ICU admission diagnosis (n=428; surgical n=276 and medical n=152), the results were similar (eTables 3 and 4) but with some loss of statistical significance in the setting of a smaller sample size and reduced power.

Table 3

Discrimination of qSOFA score and baseline risk model for in-ICU mortality among medical and surgical patients with infection

	Medical, AUROC (95% CI)	Surgical, AUROC (95% CI)	Medical v. surgical, p-value
qSOFA alone	0.60 (0.53 - 0.67)	0.60 (0.55 - 0.65)	0.32
Baseline model*	0.59 (0.51 - 0.67)	0.64 (0.59 - 0.69)	0.94
qSOFA + baseline model	0.64 (0.56 - 0.71)	0.69 (0.64 - 0.74)	0.26
	p-value
Addition of qSOFA to baseline model	0.049	<0.0001	-

qSOFA = quick Sequential Organ Failure Assessment

ICU = intensive care unit

AUROC = area under the receiver operating characteristic curve

CI = confidence interval

* Baseline model includes age, sex and HIV status

Fig. 2

qSOFA area under the receiver operating characteristic curve (AUROC) for medical v. surgical multivariable logistic regression models.

Discussion

Our study demonstrates that a higher qSOFA score at the time of ICU referral is associated with higher in-ICU mortality in patients with suspected infection in both medical and surgical patient populations in a resource-limited setting, with no statistically significant differences between groups.

The addition of qSOFA at the time of ICU referral to a baseline model known to predict in-ICU mortality (age, sex, HIV status) improved the performance of the model in both surgical and medical patients, but discrimination of all qSOFA models – dichotomous or ordinal, with and without baseline factors was poor. The use of a baseline model allows for assessment of the incremental benefit of adding qSOFA to clinical factors that are familiar to clinicians rather than a complex multivariable model.

Risk scores have traditionally been used to measure disease severity and quantify the risk of death in an ICU population rather than in an individual patient.[[20]] This allows for comparisons between clinical trials, and to benchmark quality of care between ICUs. Commonly used scores such as the Acute Physiologic and Chronic Health Evaluation (APACHE)[[21]] and the Simplified Acute Physiologic Score (SAPS)[[22]] are widely used, regularly updated, and incorporate a number of physiological variables taken within the first 24 hours following ICU admission. However, they are not easily operationalised for bedside use, especially in resource-limited settings due to the large number of inputs and requirement for laboratory-based testing, and because they incorporate some variables collected following ICU admission rather than just at admission.

Early warning systems (EWS) such as the Modified Early Warning Score (MEWS) and the National Early Warning Score (NEWS) have been developed to identify patients at risk of deterioration. These scores are also widely used and perform well in comparison to qSOFA in the prediction of in-hospital mortality and ICU transfer.[[23]] However, they are also more complex than qSOFA, and therefore are less suitable as a simple, bedside assessment tool. The Sequential (sepsis-related) Organ Failure Assessment (SOFA) score was initially designed to assess the degree of severity of organ dysfunction in septic patients.[[24]] The qSOFA score was subsequently proposed as a means to identify patients with suspected infection at risk for poor outcomes and was first validated outside of the ICU setting.[[2,25]] The early recognition of sepsis and patients with suspected infection at risk for poor outcomes is increasingly recognised as a key area of possible intervention.[[2]]

Our findings are largely consistent with recent emerging studies on qSOFA in resource-limited settings.[[13,18]] In total, these findings suggest that qSOFA can be extended from its initial validation population of non-ICU patients with infection in well-resourced settings to patients in ICUs in resource-limited settings, both with and without infection, and both medical and surgical varieties of critical illness. While qSOFA was originally developed and validated in primarily medical populations, the qSOFA components carry similar prognostic weights in both medical and surgical patients. As discrimination remains poor, qSOFA should remain one tool in a clinician’s kit to be used in context with other risk assessment and prognostic approaches. Further prospective evaluation of the role of qSOFA in these settings is advised.

Study strengths and limitations

Our study has several limitations, including its single-centre, retrospective design. Results should not be generalised unless as part of the totality of emerging evidence from multiple sites globally. Specifically, the study ICU may have lower mortality than regional peers. Our ICU is also intensivist-led, staffed with on-site registrars, and as such may represent a better-resourced environment than many lower-income settings. Finally, in the primary analysis we defined infection in any patient in whom antibiotics had been initiated at, or prior to, admission to ICU based on expert local opinion on antibiotic prescribing patterns in the study hospital and ICU. One study suggested that inappropriate empiric antibiotics were prescribed in over half of ICU admissions in SA.[[26]] This is mitigated by a strict antibiotic stewardship policy in the study ICU. But this approach may have omitted patients with undetected infection and included patients prescribed antibiotics for indications other than acute infection (e.g. prophylaxis, chronic infection). This was addressed in part by examining a highly specific subgroup of patients for whom infection was the primary ICU admission diagnosis with overall similar results.

Our study has several strengths. It is the first to our knowledge that aims to directly compare the use of qSOFA in medical and surgical patients. This may be an important differentiator given the different pathophysiological origins of infection in these populations.

A previous study compared the qSOFA with the SIRS score in a mixed population of medical and surgical patients, and showed that qSOFA had better prognostic value (AUROC 0.86 v. 0.67) for poorer outcomes.[[27]] However, there was no direct comparison to examine its utility in medical v. surgical patients.

Secondly, it supported the potential utility of qSOFA in a resource-limited setting where the more extensive laboratory work-up required for a full SOFA score calculation may not be immediately available. The qSOFA score uses only blood pressure, respiratory rate, and mentation. These parameters are available in nearly any setting using bedside assessment alone. Routinely calculating the qSOFA score when assessing a patient for possible ICU referral as a standard of care may increase experience with the score, without adding to the clinical burden. Our study specifically used qSOFA scores taken prior to in-ICU interventions.

Conclusion

A higher qSOFA score was associated with higher in-ICU mortality in medical and surgical patients with suspected infection admitted to ICU in a resource-limited setting in SA. These findings suggest that qSOFA may be useful as one tool or as an adjunct to other risk assessment and prognostic approaches to identify patients most at risk of poor outcomes.

eTables 1 - 4

eFigures 1 - 4